Reproduce “Continual Task Learning through Adaptive Policy Self-Composition” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shengchao Hu, Yuhang Zhou, Ziqing Fan, Jifeng Hu, Li Shen, Ya Zhang, Dacheng Tao
This paper presents CompoFormer, a structure-based Transformer model designed for Continual Offline Reinforcement Learning (CORL). It achieves a balance between retaining old knowledge and acquiring new skills by adaptively composing existing policies based on semantic task correlations, demonstrating superior performance on the newly introduced Offline Continual World (OCW) benchmark.
Key insights
- Traditional continual learning approaches are largely ineffective in CORL due to the unique distribution shifts inherent in combining offline reinforcement learning with sequential task acquisition.
- Adaptive policy self-composition, guided by semantic task correlations, is critical for balancing stability (preventing forgetting) and plasticity (learning new tasks) in CORL.
- Structure-based continual learning methods can be enhanced by selectively reusing and composing task-specific components, allowing for more effective knowledge transfer than simple parameter isolation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.