Reproduce “SparseLift: Automatically Optimizing Opaque Sparse Paths in Industrial-Scale Recommendation Model Training” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Changqing Shi, Anan Liu, Genbao Chen, Junchao Yang
SparseLift exposes hidden sparse operations in recommendation training so a compiler can plan their execution across embedding tables and stages. It reconstructs runtime calls as analyzable subgraphs, records execution constraints, and evaluates candidate transformations before reusing a plan during steady-state training. On 8 GPUs, median throughput rose 16.5%, from 4,334.6 to 5,050.7 samples/s, for ordinary steps in the industrial setup.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.