Reproduce “MARM: Unlocking the Future of Recommendation Systems through Memory Augmentation and Scalable Complexity” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xiao Lv, Jiangxia Cao, Shijie Guan, Xiaoyou Zhou, Zhiguang Qi, Yaqiang Zang, Ming Li, Ben Wang and 2 more

A new "cache scaling-law" for industrial recommendation systems enables multi-layer attention models to capture complex user interests by leveraging memory augmentation and scalable complexity. This approach, implemented as MARM, increased average user app usage time by +2.079% on Kuaishou's platform, demonstrating significant online performance gains with efficient resource use.

Key insights

  • A "cache scaling-law" exists for recommendation systems, demonstrating that model performance improves as cache size (L*n*d) increases, offering a new paradigm for RecSys scalability.
  • Intelligently trading off abundant storage for limited computation is a viable and cost-effective strategy to deploy more complex, deeper models in latency-sensitive industrial RecSys.
  • Different attention layers in MARM learn diversified, high-level interest expressions, allowing for sustained performance gains with increasing depth and long sequence lengths.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.