Reproduce “SignNav: Leveraging Signage for Semantic Visual Navigation in Large-Scale Indoor Environments” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jian Sun, Yuming Huang, He Li, Shuqi Xiao, Shenyan Guo, Maani Ghaffari, Qingbiao Li, Chengzhong Xu and 1 more

Researchers at the University of Macau and University of Michigan developed SignNav, a new embodied navigation task, enabling autonomous agents to interpret dynamic semantic hints from signage for navigation in large-scale indoor environments. Their Spatial-Temporal Aware Transformer (START) model achieved an 80% success rate on unseen environments, surpassing baselines, and was successfully deployed on a physical robot.

Key insights

  • The synergy of both spatial-aware and temporal-aware modules within the START model is critical for effective navigation, achieving substantial performance gains by grounding dynamic hints and maintaining long-term memory.
  • While RGB inputs are essential for semantic hint extraction, relying solely on depth observations for core navigation control resulted in superior performance and better generalization, due to depth's robustness to environmental variations.
  • Using Dataset Aggregation (DAgger) during training significantly enhances the agent's generalization capabilities by exposing it to a wider range of interactive data and its own errors, improving success rates by approximately 8%.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.