Reproduce “Sequential memory improves sample and memory efficiency in Episodic Control” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ismael T. Freire, Adrián F. Amil, Paul F.M.J. Verschure

Researchers at Radboud University developed Sequential Episodic Control (SEC), an algorithm that explicitly stores and retrieves complete behavioral sequences, enhancing both sample and memory efficiency in reinforcement learning. SEC consistently achieved superior performance across four Animal-AI tasks, demonstrating up to 1.2 times the performance of non-sequential methods with an order of magnitude less memory.

Key insights

  • Incorporating a sequential inductive bias in memory retrieval, inspired by hippocampal function, is crucial for significantly improving both sample and memory efficiency in episodic control.
  • Explicitly leveraging the temporal ordering of experiences leads to more stable policies and robust performance across a variety of cognitive tasks.
  • Forgetting mechanisms, especially reward-prioritized ones, can further enhance performance and policy stability by preventing memory stagnation and promoting adaptive memory management.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.