Reproduce “Loss of Plasticity in Continual Deep Reinforcement Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zaheer Abbas, Rosie Zhao, Joseph Modayil, Adam White, Marlos C. Machado
Researchers identified "loss of plasticity" in deep reinforcement learning agents operating in non-stationary environments, showing agents progressively lose the ability to learn new information. This phenomenon, primarily caused by "activation collapse" where neurons become inactive, was substantially mitigated by replacing standard ReLU activation functions with Concatenated ReLU (CReLU), leading to maintained learning capacity.
Key insights
- Deep RL agents (Rainbow) exhibit a progressive decline in performance and learning ability over repeated exposures to environments, termed "loss of plasticity."
- This plasticity loss is primarily caused by "activation collapse," where a growing proportion of neurons become permanently inactive (zero activation) due to the ReLU function.
- Loss of plasticity and catastrophic forgetting are distinct but related challenges, potentially requiring different solutions in continual learning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.