Reproduce “SeeNav-Agent: Enhancing Vision-Language Navigation with Visual Prompt and Step-Level Policy Optimization” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zhengcheng Wang, Zichuan Lin, Yijun Yang, Haobo Fu, Deheng Ye

SeeNav-Agent combines first-person and bird’s-eye visual prompts with SRGPO, which trains navigation decisions using local progress rewards alongside trajectory-level outcomes. On Embench-Nav, visual prompting raises GPT-4.1’s success rate to 0.867, versus 0.667 for Claude-3.5-Sonnet; Qwen2.5-VL-3B with VP+SRGPO reaches 0.723, versus 0.572 with VP+GiGPO. The process reward does not cover complex multi-floor navigation.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.