Reproduce “Mixture-of-World Models: Scaling Multi-Task Reinforcement Learning with Modular Latent Dynamics” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Boxuan Zhang, Weipu Zhang, Zhaohan Feng, Wei Xiao, Jian Sun, Jie Chen, Gang Wang
Researchers introduced Mixture-of-World Models (MoW), a modular architecture designed to scale multi-task reinforcement learning in visual environments by handling diverse observations and dynamics. The MoW agent achieved a 110.4% mean human-normalized score across 26 Atari games using 50% fewer parameters than an ensemble of single-task models, and set a new state-of-the-art for visual multi-task learning on Meta-World with a 74.5% average success rate.
Key insights
- Modular architecture, comprising task-adaptive VAEs and expert Transformers, is effective for simultaneously capturing diverse visual inputs and dynamics across heterogeneous multi-task environments.
- Gradient-based task clustering prior to training enables more meaningful parameter sharing and improved performance compared to random task partitioning for module assignment.
- The combination of harmonious loss, expert balance loss, and an auxiliary task prediction head is critical for stabilizing training, preventing expert collapse, and promoting task-discriminative feature learning in complex modular MTRL systems.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.