Reproduce “GaussMemory: Task-Driven 3D Gaussian Scene Memory for Long-Horizon Robotic Manipulation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhiqiang Hu, Shouren Huang, Masatoshi Ishikawa
GaussMemory addresses long-horizon manipulation by maintaining a persistent 3D scene record and learning which object-level information to read and update for each task. Its unified memory attention couples task-conditioned readout with selective updates to stored 3D Gaussians. On LIBERO Long-10, it achieves 94.1% success versus 91.8% for a decoupled baseline, but this does not isolate coupling alone.
Key insights
- An 'active, task-driven' spatial memory, where the robot learns *how* to remember (e.g., which objects to update aggressively), is superior to passive, rule-based memory systems for long-horizon tasks.
- Explicit 3D geometric representation, provided by 3D Gaussian Splatting, is crucial for achieving high performance in spatially demanding manipulation tasks, offering precise object localization and tracking over time.
- Unifying memory update and readout into a single, differentiable attention mechanism allows task loss to directly sculpt memory management behavior, leading to emergent intelligent strategies like differential update rates for manipulated vs. static objects.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.