Reproduce “TRAVEL: Training-Free Retrieval and Alignment for Vision-and-Language Navigation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Navid Rajabi, Jana Kosecka

Researchers at George Mason University introduced TRAVEL, a training-free, modular framework for Vision-and-Language Navigation (VLN) that leverages the zero-shot capabilities of Large Language Models (LLMs) and Vision-Language Models (VLMs). This approach achieved 70.0% Precision@10 for last landmark localization and an average normalized Dynamic Time Warping (nDTW) score of 88.92% on the complex R2R-Habitat dataset.

Key insights

  • Modern VLMs like SigLIP achieve significantly superior last landmark localization performance (70.0% Precision@10) on panoramic views compared to previous methods using CLIP on semantic occupancy maps (34.4%).
  • A modular, training-free architecture leveraging zero-shot LLMs and VLMs can effectively tackle complex VLN tasks, offering improved interpretability and adaptability without task-specific training.
  • Explicit, individual landmark grounding combined with dynamic programming alignment (PANO2LAND) yields slightly better performance than direct VLM rating, highlighting the benefit of structured decomposition for complex path evaluation.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.