Reproduce “Curriculum Reinforcement Learning for Complex Reward Functions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kilian Freitag, Kristian Ceder, Rita Laezza, Knut Åkesson, Morteza Haghir Chehreghani
Researchers at Chalmers University of Technology developed a two-stage reward curriculum for reinforcement learning that enables agents to effectively learn in environments with complex, multi-term reward functions. This approach progressively introduces constraints, leading to policies that better balance task completion with constraint satisfaction, consistently outperforming single-stage training methods in simulated robotics tasks.
Key insights
- Training with a simplified base reward first prevents agents from exploiting individual constraint terms and getting stuck in suboptimal local optima for the overall task.
- Pre-training the agent's network with the base reward in the initial phase creates a beneficial prior that significantly aids in learning the full, complex reward function efficiently.
- An automatic mechanism for switching between curriculum phases provides higher sample efficiency and adaptability compared to fixed-time switches.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.