Reproduce “Dynamic Weights in Multi-Objective Deep Reinforcement Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Axel Abels, Diederik M. Roijers, Tom Lenaerts, Ann Nowé, Denis Steckelmacher
This research introduces a Conditioned Network (CN) architecture and Diverse Experience Replay (DER) to enable Deep Reinforcement Learning agents to adapt to dynamically changing objective preferences in multi-objective environments. The proposed methods achieve superior performance compared to existing approaches in visual-input, stochastic environments like the newly proposed Minecart benchmark, demonstrating robust adaptation and efficient experience management.
Key insights
- A single Conditioned Network architecture can effectively generalize across varying objective preferences by incorporating the weight vector as input, enabling faster adaptation than maintaining multiple distinct networks.
- A dual-loss training approach, considering both the current active weight vector and a randomly sampled past weight vector, is crucial for balancing adaptation to new preferences and maintaining knowledge of previously learned policies.
- Diverse Experience Replay, which stores and samples entire trajectories based on their reward diversity, mitigates replay buffer bias and catastrophic forgetting in non-stationary multi-objective environments, especially when objective preferences change sparsely.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.