Reproduce “Unleashing the Potential of Sparse Attention on Long-term Behaviors for CTR Prediction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Weijiang Lai, Beihong Jin, Di Zhang, Siru Chen, Jiongyan Zhang, Yuhang Gou, Jian Dong, Xingxing Wang
SparseCTR predicts click-through rates from user histories by splitting timelines at large time gaps and combining attention to chunk summaries, recent transitions, and nearby behaviors. It also encodes elapsed time and calendar patterns. On the industrial dataset, it reports an offline AUC of 0.7083 versus 0.7040 for SUAN; this comparison does not establish which component caused the difference.
Key insights
- Tailoring sparse attention and temporal encoding to the unique personalized and temporal dynamics of user behavior data is crucial for efficiently enhancing CTR prediction performance.
- Modeling user interests at multiple granularities (global, transition, local) and performing personalized, time-aware sequence segmentation are effective strategies for processing extensive and complex user interaction histories.
- The SparseCTR model demonstrates a clear scaling law, where predictive performance consistently improves with increased computational resources, indicating its potential for predictable performance gains at larger scales.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.