Reproduce “GRACE: Generative Recommender Acceleration Engine for Real-Time Ads Retrieval” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhou Fang, Yuhang Huang, Ang Zhang, Yihan He, Ruichao Xiao, Chao Li, Yavuz Yetim, Sibyl Yang and 7 more
Meta Platforms developed GRACE, a serving system that productionizes generative recommenders for real-time ads retrieval by integrating personalized eligibility checks directly into the decoding process and implementing specialized kernel optimizations. The system achieved an ad-level target matching pass rate increase from 23.55% to 40.42% and delivered substantial kernel speedups, such as a 68.0x improvement for cross-attention.
Key insights
- Directly integrating personalized eligibility checks into the generative decoding loop (Generative Target Matching) is critical for improving the efficiency of real-time ads retrieval by preventing the generation of ineligible candidates.
- Customizing attention kernels and KV cache management for the 'wide-beam, short-sequence' computational profile of generative ads recommenders leads to substantial latency reductions and hardware utilization improvements.
- Dynamic adjustment of beam sizes during the decoding process can further optimize computational resources, especially during initial decoding steps.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.