Reproduce “AllDayNav: Lifelong Navigation via Real-World Reinforcement Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Hang Yin, Yinan Liang, Jiazhao Zhang, Jiahang Liu, Minghan Li, Zhizheng Zhang, He Wang

AllDayNav is a framework for lifelong embodied navigation, combining a self-evolving multimodal memory with real-world reinforcement learning to enable autonomous robots to learn and adapt in dynamic, unknown indoor environments over extended periods. It demonstrates consistent performance improvements in simulation and achieves high success rates, such as 96% in a laboratory and 97% in a living room, on a quadruped robot in real-world settings.

Key insights

  • Integrating a self-evolving multimodal memory with policy learning allows the implicit encoding of scene dynamics and continuous knowledge accumulation for lifelong navigation.
  • Autonomous generation of diverse, contextually relevant open-vocabulary tasks, paired with hybrid goal retrieval, is crucial for driving effective policy optimization without human supervision.
  • Combining Conservative Q-Learning with a Behavior Cloning loss effectively stabilizes online learning, preventing catastrophic forgetting of pretrained skills and facilitating robust adaptation in dynamic environments.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.