Reproduce “SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hang Yin, Xiuwei Xu, Zhenyu Wu, Jie Zhou, Jiwen Lu
SG-Nav introduces a framework for zero-shot object navigation that enables robots to find objects in unfamiliar settings using natural language instructions. The system combines online 3D scene graph construction with hierarchical chain-of-thought prompting for LLM reasoning and a re-perception mechanism to handle perception errors, achieving state-of-the-art performance across multiple benchmarks.
Key insights
- Combining a hierarchical 3D scene graph with large language model reasoning enables robust zero-shot object navigation in unfamiliar environments.
- Hierarchical Chain-of-Thought prompting guides LLMs to perform spatial reasoning and make navigation decisions by breaking down complex environment understanding into manageable subgraphs.
- A re-perception mechanism enhances robustness by verifying object detections, mitigating errors from perception systems.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.