Reproduce “Retrieval-Augmented Embodied Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yichen Zhu, Zhicai Ou, Xiaofeng Mou, Jian Tang
The RAEA framework augments embodied agents by integrating an external policy memory bank for explicit knowledge retrieval, facilitating efficient learning and generalization. It consistently achieves superior performance in few-shot learning across simulated tasks and real-world robot manipulation, demonstrating improved success rates, such as 62.7% for PickCube with dual-modal input.
Key insights
- Explicit memory retrieval via a policy memory bank significantly enhances embodied agent performance and generalization, particularly in data-limited, few-shot learning scenarios.
- Leveraging multi-modal inputs (text, audio, image, point cloud) substantially improves a robot's ability to interpret instructions and perceive environments, leading to increased generalizability.
- Incorporating cross-embodiment data from diverse robot platforms into the policy memory bank facilitates transfer learning and boosts performance, even for different target robots.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.