Reproduce “Merlin HugeCTR: GPU-accelerated Recommender System Training and Inference” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Joey Wang, Yingcan Wei, Minseok Lee, Matthias Langer, Fan Yu, Jie Liu, Alex Liu, Daniel Abel and 4 more
Merlin HugeCTR provides an open-source, GPU-accelerated framework for recommender system training and inference, efficiently handling terabyte-scale embedding tables. The NVIDIA framework achieved up to a 24.6x speedup in DLRM training on the MLPerf v1.0 benchmark and a 5x to 62x speedup for online inference using a Hierarchical Parameter Server compared to CPU baselines.
Key insights
- A hybrid parallelization strategy combining Model Parallelism for sparse embeddings and Data Parallelism for dense layers effectively addresses the distinct computational and memory demands of recommender systems on GPUs.
- Hierarchical memory management, spanning GPU GDDR, CPU system memory, and persistent storage, is essential for accommodating and efficiently accessing terabyte-scale embedding tables during both training and low-latency inference.
- Overlapping parameter updates and data migrations with dense model computations, alongside online propagation of embedding updates, enables continuous learning and minimizes inference latency in dynamic environments.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.