Reproduce “MEMORA: Embodied Action Memory from Egocentric Videos for Reasoning and Planning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zihao Yu, Xiu Yuan, Chongjie Zhang
The MEMORA framework enables embodied agents to form and utilize persistent Embodied Action Memory (EAM) from continuous egocentric videos through a multi-store architecture, online active editing, and offline consolidation. It significantly improves an agent's reasoning and planning abilities, especially for generalization tasks, demonstrating up to a +16.4% relative planning gain on unseen scenarios.
Key insights
- Online active memory editing can achieve substantial compression (median 18x reduction in entity records) while preserving crucial information for memory faithfulness.
- Offline consolidation is critical for converting episodic observations into generalizable knowledge, yielding significant gains in cross-event reasoning and planning for novel tasks.
- A multi-store memory architecture that functionally separates and manages distinct information types (e.g., static environment vs. dynamic entities vs. inferred knowledge) is effective for handling the complexity of embodied experience.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.