Reproduce “RADIO-ViPE: Online Tightly Coupled Multi-Modal Fusion for Open-Vocabulary Semantic SLAM in Dynamic Environments” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zaid Nasser, Mikhail Iumanov, Tianhao Li, Maxim Popov, Jaafar Mahmoud, Sergey Kolyubin
RADIO-ViPE is an online system for open-vocabulary semantic SLAM that processes uncalibrated monocular RGB video, constructing a robust and semantically-aware 3D map in dynamic environments. It achieves an average ATE of 1.63 cm on TUM-RGBD dynamic sequences and competitive 3D semantic segmentation on Replica, operating at 8-10 FPS.
Key insights
- Jointly optimizing high-level vision-language embeddings with geometric constraints within a factor graph enables robust, semantically consistent 3D mapping from monocular video.
- Aggregating temporal evidence to dynamically adjust robust kernel parameters effectively differentiates and handles static surfaces, movable objects, and actively moving agents in dynamic scenes.
- Real-time, open-vocabulary semantic SLAM from uncalibrated monocular input is achievable by leveraging foundation models for dense features and employing robust multi-modal optimization.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.