Reproduce “TWIN: TWo-stage Interest Network for Lifelong User Behavior Modeling in CTR Prediction at Kuaishou” on OpenResearch

About this paper

Jianxin Chang, Chenbin Zhang, Zhiyi Fu, Xiaoxue Zang, Lin Guan, Jing Lu, Yiqun Hui, Dewei Leng and 3 more

TWIN makes a short-video recommender’s first-stage filter use its final scorer’s target-attention relevance measure, while simplifying long-history computation. It returns 100 behaviors. TWIN reaches sample-weighted GAUC 0.7336 versus 0.7299 for SIM Soft; AUC tests whether positives outrank negatives, and the reported improvement is 0.51%. Its 94-of-100 retrieval result came from tiny demo data, not production-scale evaluation.

Key insights

• Ensuring consistency in the relevance evaluation logic between the coarse-grained retrieval (GSU) and fine-grained attention (ESU) units significantly improves the quality of retrieved behaviors for lifelong user modeling.
• The computational overhead of attention mechanisms for extremely long sequences can be mitigated by strategically splitting features and leveraging offline pre-computation and caching for stable components.
• User-item cross features play a vital role in capturing specific behavioral context, and their effective incorporation, even through simplified bias terms, enhances attention-based relevance scoring.

Summary by alphaXiv.