Reproduce “EmbodiedSplat: Personalized Real-to-Sim-to-Real Navigation with Gaussian Splats from a Mobile Device” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Gunjan Chhablani, Xiaomeng Ye, Muhammad Zubair Irshad, Zsolt Kira
EmbodiedSplat introduces a pipeline that uses low-effort mobile phone video captures and 3D Gaussian Splatting to create high-fidelity simulated environments for embodied agent training. This approach enables personalized policy fine-tuning for image-goal navigation, showing a strong sim-to-real correlation and improved real-world performance on a Stretch robot.
Key insights
- High-quality 3D meshes suitable for embodied navigation policy training can be efficiently generated from casual mobile phone captures using 3D Gaussian Splatting.
- Personalized fine-tuning of pre-trained policies on these environment-specific reconstructed meshes substantially improves both simulated and real-world navigation performance, particularly for new and complex environments.
- The pipeline achieves a robust sim-to-real correlation (0.87-0.97 SRCC), indicating that performance improvements in the reconstructed simulation environments reliably translate to gains in real-world robot deployment.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.