Reproduce “FuXi-$α$: Scaling Recommendation Model with Feature Interaction Enhanced Transformer” on OpenResearch

About this paper

Yufei Ye, Wei Guo, Jin Yao Chin, Hao Wang, Hong Zhu, Xi Lin, Yuyang Ye, Yong Liu and 3 more

A Transformer-based recommendation model, FuXi-𝛼, demonstrates that sequential recommendation models can adhere to scaling laws, consistently improving performance with increased size. It achieves state-of-the-art results on multiple benchmarks and shows a 4.67% increase in songs played and a 5.10% increase in listening duration in a real-world A/B test within the Huawei Music app.

Key insights

• Sequential recommendation models can adhere to scaling laws, similar to large language and vision models, with performance improving as model size, data, and computation increase.
• Explicitly disentangling semantic, temporal, and positional information within self-attention is crucial for capturing richer sequential dependencies.
• Robustly modeling implicit feature interactions through a multi-stage feed-forward network is as important as explicit interactions for maximizing a Transformer's expressiveness in recommendation.

Summary by alphaXiv.