Reproduce “Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhaojie Gong and 4 more
The authors introduce Generative Recommenders, which model content and user actions as a sequence for both candidate ranking and next-content retrieval. Their HSTU encoder supports long histories and target-aware scoring, while candidate batching improves serving throughput. In internal production experiments, the system reports online gains of 12.4% on engagement and 4.4% on consumption versus a confidentially configured DLRM baseline.
Key insights
- Power-law scaling laws, previously observed in large language models, extend to recommendation systems, enabling continuous quality improvement with increased compute, data, and parameters.
- Reformulating heterogeneous recommendation features into a unified sequential modality allows for powerful generative modeling that can capture complex user intent more effectively than discriminative DLRMs.
- Specialized architectural designs (like HSTU's pointwise aggregated attention) and algorithmic innovations (e.g., SL, M-FALCON) are crucial for efficiently deploying trillion-parameter sequential models in production.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.