Reproduce “Privileged Features Distillation at Taobao Recommendations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chen Xu, Quan Li, Junfeng Ge, Jinyang Gao, Xiaoyong Yang, Changhua Pei, Fei Sun, Jian Wu and 2 more
Privileged Features Distillation (PFD) transfers information from post-event or otherwise unavailable features into a recommendation model that uses only serving-time inputs. Evaluated on Taobao CTR and CVR ranking tasks, PFD and its combination with model distillation improved offline AUC, delivered online gains of 5.0% in click metrics and 2.3% in conversion metrics, and added as little as 3.6%–11.4% training time through shared computation.
Key insights
- A teacher that sees regular features alongside privileged ones is more useful than one relying on privileged features alone, because contextual information helps interpret signals such as dwell time or post-click behavior.
- PFD provides a single distillation target for many privileged features, avoiding the loss-weighting and task-interference issues associated with modeling each auxiliary feature separately.
- Deployment constraints shape the model design: separating user and item representations enables efficient coarse ranking, while selectively unsharing the student’s user embedding recovers performance lost through excessive parameter sharing.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.