Reproduce “CLM: Removing the GPU Memory Barrier for 3D Gaussian Splatting” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hexu Zhao, Xiwen Min, Xiaoteng Liu, Moonjun Gong, Yiming Li, Ang Li, Saining Xie, Jinyang Li and 1 more
CLM is a system designed to overcome the GPU memory limitations of 3D Gaussian Splatting, enabling the training of models up to 6.1 times larger on a single consumer-grade GPU. This approach uses intelligent CPU memory offloading and pipelining to achieve higher reconstruction quality for complex 3D scenes while maintaining high throughput.
Key insights
- 3DGS computations exhibit significant sparsity, as only a small subset of Gaussians within a camera's frustum are actively processed for any given view.
- Sparsity patterns across camera views display spatial locality, meaning adjacent views access largely overlapping sets of Gaussians, which can be leveraged for efficient caching.
- Frustum culling, a crucial step, requires only a minimal subset of each Gaussian's attributes, allowing it to be performed efficiently on the GPU even when most data is offloaded to CPU memory.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.