Reproduce “Smart-Infinity: Fast Large Language Model Training using Near-Storage Processing on a Real System” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hongsun Jang, Jaeyong Song, Jaewon Jung, Jaeyoung Park, Youngsok Kim, Jinho Lee
Researchers at Seoul National University developed Smart-Infinity, a framework that accelerates large language model training by leveraging near-storage processing on multiple commercial computational storage devices. This approach, which offloads parameter updates and gradient compression, achieves up to a 2.11x speedup compared to a software RAID0 baseline by mitigating the storage bandwidth bottleneck.
Key insights
- Near-Storage Processing effectively bypasses the PCIe bandwidth saturation limit, enabling linear scaling of storage throughput for LLM training.
- Offloading data-intensive but computationally lightweight parameter updates to CSD FPGAs drastically reduces data movement over the main system interconnect.
- Careful design of CSD-internal data handling, including pre-allocation and swap overlapping, is essential for realizing maximum performance benefits on real hardware.
- Combining gradient compression with CSD-side decompression further enhances scalability by reducing remaining gradient transfer bottlenecks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.