Reproduce “Deep Episodic Memory: Encoding, Recalling, and Predicting Episodic Experiences for Robot Action Execution” on OpenResearch

About this paper

Jonas Rothfuss, Fabio Ferreira, Eren Erdal Aksoy, You Zhou, Tamim Asfour

Researchers developed a deep neural network architecture that functions as an episodic memory for robots, encoding visual action experiences into a latent space, and enabling their reconstruction, future prediction, and retrieval based on conceptual similarity. This system allows robots to learn and execute manipulation tasks by leveraging past demonstrations to adapt to novel objects and scenes.

Key insights

• Unsupervised training on raw visual data can yield subsymbolic latent representations where conceptual similarity between different action episodes is inherently preserved, enabling effective retrieval and comparison.
• A single, compact latent vector generated from an initial sequence of video frames is sufficient to reconstruct the original visual episode and predict its subsequent frames, demonstrating the latent space's comprehensive capture of both static scene content and dynamic action evolution.
• The deep episodic memory framework facilitates robust case-based reasoning in robotics, allowing a robot to transfer motion knowledge from a single recalled demonstration to perform a manipulation task on a novel object in a different scene.

Summary by alphaXiv.