Reproduce “Context-Aware Planning and Environment-Aware Memory for Instruction Following Embodied Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Byeonghwi Kim, Jinyeon Kim, Yuyeong Kim, Cheolhong Min, Jonghyun Choi
This paper from Yonsei University introduces CAPEAM, a framework that enhances embodied agents' ability to follow complex instructions by integrating Context-Aware Planning (CAP) and Environment-Aware Memory (EAM). CAPEAM achieves state-of-the-art performance on the ALFRED benchmark, demonstrating up to a +10.70% increase in success rate in unseen environments.
Key insights
- Explicitly defining and utilizing a 'context' of task-relevant objects throughout the planning hierarchy significantly improves an agent's ability to focus on correct interactions.
- A robust, dynamic memory system capable of tracking object relocation, state changes, and past visual masks is essential for agents operating in interactive environments.
- Combining a high-level context-aware planner with a detailed action planner and an environment-aware memory module leads to synergistic improvements in instruction following and generalization.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.