Reproduce “Repetition as Reinforcement: Enhancing Sample Efficiency via Instant Episode Repetition in Reinforcement Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Hoda Yamani, Yuning Xing, Koen van Rijnsoever, Bruce A. MacDonald, Henry Williams

Researchers at the University of Auckland developed Instant Episode Repetition (IER), a biologically inspired mechanism that actively re-executes high-reward action sequences directly during reinforcement learning data collection. The method consistently improved sample efficiency and accelerated policy convergence across eight continuous control benchmarks and a real-world robotic task, outperforming standard baselines and Self-Imitation Learning variants.

Key insights

  • Re-executing successful action sequences under new initial conditions generates diversified, yet related, trajectories that increase sampling density in high-reward state-action regions.
  • This active, episode-level repetition facilitates behavioral consolidation similar to biological learning, strengthening memory and stabilizing skill acquisition.
  • IER's effectiveness is robust in stochastic real-world environments, indicating its ability to handle noise and uncertainties through varied re-executions.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.