Reproduce “Balancing Plasticity and Stability with Fast and Slow Successor Features” on OpenResearch

About this paper

Raymond Chua, Doina Precup, Blake Richards

A deep reinforcement learning framework, combining Successor Features with a multi-timescale synaptic consolidation mechanism (SF+SC), was developed to enable robust adaptation in environments with naturalistic, continually drifting dynamics. This framework consistently outperformed baseline methods in maintaining performance under gradual environmental changes.

Key insights

• Under naturalistic, gradual environmental non-stationarity, maintaining stability (retaining old knowledge) is a more critical performance bottleneck than maximizing plasticity (adapting to new information).
• Consolidating predictive representations like Successor Features proves more effective for robust adaptation to changing dynamics than consolidating traditional Q-value parameters.
• A multi-timescale synaptic consolidation system, particularly with more distinct timescale variables, enhances learning efficiency by allowing the preservation of stable, long-term knowledge while enabling immediate adaptation.

Summary by alphaXiv.