Reproduce “ViNL: Visual Navigation and Locomotion Over Obstacles” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Simar Kareer, Naoki Yokoyama, Dhruv Batra, Sehoon Ha, Joanne Truong
A hierarchical visual policy, ViNL, enables quadrupedal robots to skillfully navigate unseen indoor environments and intelligently step over small obstacles, a capability demonstrated by achieving a 73.6% success rate in cluttered settings and reducing foot collisions by 4.42 per meter.
Key insights
- A multi-stage training curriculum is crucial for effective visual locomotion over clutter, where pre-training on rough terrain before fine-tuning for obstacle avoidance improves success rates by 19.2%.
- Zero-shot sim-to-sim transfer between different simulators (Habitat and Isaac Gym) and control abstractions (kinematic and rigid-body dynamics) is viable, showing a small 2.9% drop in navigation success rate.
- Incorporating temporal memory (LSTM) in the visual locomotion policy is essential for superior performance, enabling the robot to 'remember' obstacles and yielding a 6.8% higher success rate than an MLP-based approach.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.