Reproduce “General-Purpose Robotic Navigation via LVLM-Orchestrated Perception, Reasoning, and Acting” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Bernard Lange, Anil Yildiz, Mansur Arief, Shehryar Khattak, Mykel Kochenderfer, Georgios Georgakis

Stanford University and JPL researchers developed the Agentic Robotic Navigation Architecture (ARNA), which employs a Large Vision-Language Model to dynamically orchestrate a comprehensive robotic stack for general-purpose navigation in unmapped environments. The system achieved 77% accuracy and a mean path length of 16.55 meters on the HM-EQA benchmark, surpassing existing baselines while demonstrating adaptability to diverse natural language tasks.

Key insights

  • Large Vision-Language Models (LVLMs) can serve as effective orchestrators for entire robotic stacks, dynamically generating task-specific workflows to replace rigid, expert-designed pipelines.
  • Grounding LVLM reasoning with multimodal memory, including a 3D scene graph and an occupancy grid map, is critical for efficient spatial awareness, preventing hallucinations, and enabling targeted exploration.
  • A structured, self-discover-inspired approach for workflow generation significantly improves decision-making and tool management compared to simpler Chain-of-Thought prompting.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.