Reproduce “LG-VLN: A Zero-Shot Vision-and-Language Navigation Framework with LangGraph State Orchestration” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jianhe Zhao, Yanhua Qiu, Zhiyu Zhang, Zibo Zhao, Jinhua Xie
LG-VLN uses monocular RGB, CleanDIFT features, and BLIP-2 image–text relevance to reconstruct maps, guide waypoints, and retain navigation state across recovery. On a fixed 550-episode R2R-CE subset in Habitat-Sim, it achieves 21.3% SR and 12.1% SPL, versus CA-Nav’s 22.4% and 9.3% with RGB-D and odometry. Removing CleanDIFT or BLIP-2 from the value map lowers SR to 3.0% or 12.0%, respectively. These simulation results do not establish a causal benefit from LangGraph.
Key insights
- Using one dense visual representation for both geometry and semantics substantially improves target localization: removing CleanDIFT reduces success from 21.3% to 3.0%, while removing BLIP-2 lowers it to 12.0%.
- Pose estimation is more damaging than depth estimation in the reported ablation: predicted depth with ground-truth poses reaches 24.2% success, compared with 21.3% when both depth and poses are predicted.
- Explicit state orchestration turns navigation recovery into a persistent, inspectable process: temporary planning failures can be reset while retaining the global map and active instruction.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.