Reproduce “Grounded Language Learning Fast and Slow” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Felix Hill, Olivier Tieleman, Tamara von Glehn, Nathaniel Wong, Hamza Merzic, Stephen Clark
DeepMind researchers developed an embodied AI agent capable of fast-mapping, allowing it to quickly learn and utilize novel word-object associations in a 3D environment. The proposed Dual-coding Episodic Memory architecture enabled robust one-shot language acquisition, strong generalization to new objects and categories, and seamless integration with slowly acquired skills.
Key insights
- Explicitly disentangling and aligning multi-modal (linguistic and visual) information within a specialized memory architecture is crucial for successful grounded fast-mapping.
- Advanced memory systems, specifically DCEM, proved more memory-efficient and capable of retaining episodic information over time compared to Transformer-based architectures, especially under capacity constraints.
- Incorporating intrinsic motivation signals derived from language-novelty within the dual-coding memory enables agents to learn fast-mapping without explicit shaping rewards.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.