Reproduce “Adaptive World Memory 3D Foundation Model for Scalable 3D Mapping, Localization, and Rendering” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tianchen Deng, Guole Shen, Yilin Shen, Wenhua Wu, Yilin Fang, Ziqi Ma, Tianjun Zhang, Shenghai Yuan and 2 more
AWM-3DFM turns streaming RGB views into an indoor–outdoor map by selectively updating learned memory with temporal and spatial cues, then globally aligning local submaps. It decodes memory into geometry and renderable 3D Gaussians. On Apartment, combined cues yield 9.14 cm accuracy, 7.30 cm completeness, and 0.38 m ATE RMSE; five-sequence average ATE RMSE is 0.50 m versus 0.57 m for MASt3R-SLAM in tested settings.
Key insights
- Token-level memory control and map-level optimization address different failure modes: adaptive regulation limits local memory contamination, while submaps and loop closure reduce long-range drift.
- Using submap-coordinate pointmaps directly as Gaussian centers connects metric reconstruction with efficient novel-view synthesis without a separate depth back-projection pipeline.
- The ablations show complementary benefits from temporal and spatial regulation: on the Apartment dataset, adding both reduced reconstruction accuracy error from 20.40 cm to 9.14 cm and ATE from 0.70 m to 0.38 m before the full global configuration.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.