Reproduce “DreamNav: A Trajectory-Based Imaginative Framework for Zero-Shot Vision-and-Language Navigation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yunheng Wang, Yuetong Fang, Taowen Wang, Yixiao Feng, Yawen Tan, Shuning Zhang, Peiran Liu, Yiding Ji and 1 more
A framework named DreamNav enables zero-shot Vision-and-Language Navigation in continuous environments by unifying trajectory-level planning with active imagination, relying solely on low-cost egocentric visual inputs. It achieved a 1.79% higher Success Rate and 4.95% higher Success-weighted Path Length compared to the strongest panoramic baseline in simulated environments and superior robustness in real-world tests.
Key insights
- Effective zero-shot Vision-and-Language Navigation can be achieved using only low-cost egocentric inputs when combined with robust viewpoint stabilization.
- Trajectory-level planning offers better semantic alignment with natural language instructions and enables more coherent, long-horizon navigation compared to point-level decisions.
- Active imagination, specifically abstracted into semantic narratives by foundation models, provides crucial foresight for proactive reasoning and optimal path selection without explicit environmental exploration.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.