Reproduce “Learning From Failures: Efficient Reinforcement Learning Control with Episodic Memory” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chenyang Miao
Researchers from Xi'an Jiaotong University developed Failure Episodic Memory Alert (FEMA), a reinforcement learning technique that utilizes prematurely terminated "failure" trajectories to guide exploration away from hazardous states. FEMA consistently improved sample efficiency, convergence speed, and final performance for SAC, PPO, and CrossQ algorithms on robotic control benchmarks and enabled a real bipedal robot to climb stairs.
Key insights
- Prematurely terminated "failure" trajectories, often discarded, contain valuable spatiotemporal patterns indicating state-action sequences that lead to undesirable outcomes.
- Explicitly learning from and leveraging these failure experiences can effectively guide an agent away from previously encountered hazardous states, promoting more stable and effective exploration.
- A "failure-centric" approach to episodic memory can significantly improve the sample efficiency, convergence speed, and final performance of various model-free reinforcement learning algorithms in challenging robot control tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.