Reproduce “Continual Backprop: Stochastic Gradient Descent with Persistent Randomness” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shibhansh Dohare, Richard S. Sutton, A. Rupam Mahmood
Researchers empirically demonstrate that standard Backprop suffers from "decaying plasticity" in non-stationary environments, losing its ability to adapt over time. They introduce Continual Backprop (CBP), which consistently maintains learning performance across thousands of non-stationarities in supervised and reinforcement learning tasks.
Key insights
- Standard Backprop's initial success in non-stationary problems stems from its randomly initialized weights, which confer "plasticity," but this plasticity decays over time as weights are optimized.
- The "decaying plasticity" is a fundamental temporal asymmetry in Backprop, distinct from catastrophic forgetting, and represents a core limitation for perpetual learning in dynamic environments.
- Continuously replacing low-utility features with new randomly initialized ones, guided by a utility measure that balances contribution and adaptability, can effectively combat decaying plasticity.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.