Reproduce “Zero-shot Cross-domain Knowledge Distillation: A Case study on YouTube Music” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Srivaths Ranganathan, Nikhil Khani, Shawn Andrews, Chieh Lo, Li Wei, Gergo Varady, Jochen Klingenhoefer, Tim Steele and 3 more
Google LLC researchers applied zero-shot cross-domain knowledge distillation to improve multi-task ranking models in YouTube Music, a low-traffic recommender system. The method transferred knowledge from a large-scale YouTube video recommendation model, leading to substantial online gains in user engagement and new content discovery.
Key insights
- Knowledge distillation can be effective even if the zero-shot teacher model's direct predictions on the target domain exhibit lower accuracy compared to a baseline model trained solely on target data.
- Distilling knowledge for a subset of tasks, or even for non-serving auxiliary tasks, can lead to performance improvements in other, non-distilled tasks within the multi-task student model.
- The methodology notably enhances the student model's ability to generalize and adapt to new content and trends, indicated by significant online uplift in content discovery and engagement with new releases.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.