Reproduce “FloorPlan-VLN: A New Paradigm for Floor Plan Guided Vision-Language Navigation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Kehan Chen, Yan Huang, Dong An, Jiawei He, Yifei Su, Jing Liu, Nianfeng Liu, Liang Wang

FloorPlan-VLN introduces a new paradigm for Vision-Language Navigation where embodied agents utilize structured semantic floor plans as global spatial priors. The FP-Nav model, based on Qwen-2.5-VL, enables navigation with concise, region-level instructions and demonstrated robustness to noise, achieving 20.9% success rate and 15.3% SPL on the FloorPlan-R2R Val-Unseen split.

Key insights

  • Leveraging pre-existing global spatial priors, such as floor plans, allows for more human-like, efficient navigation using concise, high-level instructions.
  • A spatio-temporally aligned dual-view input, explicitly rendering historical trajectory and current pose on a dynamic floor plan alongside egocentric observations, significantly enhances cross-modal perception and global-local alignment.
  • Auxiliary tasks (Region Localization, Trajectory Reasoning, Instruction Summarization) are crucial for strengthening an MLLM's ability to perform spatio-semantic alignment, region-level planning, and deeper understanding of goal-oriented language.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.