Reproduce “GASE: Gaussian Splatting-Based Automated System for Reconstructing Embodied-Simulation Environments” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiawei Zhang, Yiming Yan, Chao Liang, Nuo Xu, Seson Sun, Qichen Zhang, Yuhao Xu, Yantai Yang and 3 more
The GASE system reconstructs high-fidelity embodied simulation environments by leveraging multi-view data acquisition and a 2D-domain object-scene decomposition pipeline. It achieves over 10% higher segmentation accuracy than 3DGS methods and enables real-robot manipulation with a sim-to-real gap under 10%, while also improving navigation success rates.
Key insights
- Performing object-scene decomposition in the 2D image domain significantly improves mask accuracy and inpainting quality compared to direct 3D Gaussian segmentation.
- Combining multi-view data acquisition with a camera-pose-based localization strategy and video propagation enables robust and comprehensive object mask extraction.
- Decoupling static background (with collision mesh) and individual foreground objects creates realistic, manipulable assets for versatile embodied AI training.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.