Reproduce “RAVEN: Long-Horizon Reasoning & Navigation with a Visuo-Spatio-Temporal Memory” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yixun Hu, Zhicheng Zheng, Lihan Zha, Chunwei Xing, Rajdeep Singh, Omar Hossain, Antonio Loquercio, Dhruv Shah
RAVEN introduces a visuo-spatio-temporal memory system for robots, leveraging direct visual embeddings from foundation models to bypass the captioning bottleneck for long-horizon reasoning and navigation. The system achieves superior performance against caption-based methods, demonstrates high memory efficiency, and successfully deploys on a physical robot for complex natural language goal-reaching tasks.
Key insights
- Direct visual embeddings robustly preserve fine-grained semantic details that are often lost or inaccurately described by text-based captioning methods.
- Providing similarity scores with retrieved memory items and ordering temporal memories chronologically significantly enhances the VLM's reasoning capabilities.
- RAVEN offers a training-free, computationally lightweight, and plug-and-play memory system that is highly compact and scalable for long-horizon embodied AI tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.