Reproduce “Robostral Navigate” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Abdelaziz Bounhar, Abhijeet Somani, Aditi Kabra, Adrian Valente, Adrien Petralia, Adrien Sade, Alan Jeffares, Albert Jiang and 268 more
Robostral Navigate uses a single monocular RGB camera for instruction-guided robot navigation. Its vision-language model points to a visible waypoint in the image, or predicts local metric displacements when the destination is out of view; a separate policy generates actions. On R2R-CE validation-unseen, it reaches 77.4% success, above listed monocular and depth/multi-camera baselines, a whole-system benchmark comparison.
Key insights
- High-performance, state-of-the-art embodied navigation can be achieved using only a single monocular RGB camera, challenging the reliance on more complex and costly sensor modalities.
- Predicting navigation waypoints directly in the camera's image space, combined with extensive randomization of robot configurations during training, enables robust generalization across various robot embodiments and camera intrinsics.
- A hybrid training approach, combining highly efficient supervised fine-tuning with online reinforcement learning, effectively mitigates exposure bias and substantially improves a navigation policy's robustness and success rate in unseen environments.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.