Reproduce “LREA: Low-Rank Efficient Attention on Modeling Long-Term User Behaviors for CTR Prediction” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xin Song, Xiaochen Li, Jinxin Hu, Hong Wen, Zulong Chen, Yu Zhang, Xiaoyi Zeng, Jing Zhang

LREA models extensive long-term user behaviors for Click-Through Rate (CTR) prediction by utilizing low-rank matrix decomposition and matrix absorption. This approach achieves a 5.88% CTR gain and 4.26% RPM gain in live A/B tests, while maintaining low inference latency with only a 3 ms increase.

Key insights

  • It is possible to maintain high accuracy by modeling all long-term user behaviors while achieving real-time inference efficiency, overcoming the traditional performance-efficiency trade-off.
  • Pre-computing and caching compressed, absorbed representations allows high-dimensional computations to be shifted offline, drastically reducing online latency.
  • A pragmatic application of ReLU's properties, coupled with a non-negativity regularization, enables efficient matrix absorption in a non-linear deep learning context.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.