Reproduce “Open-Ended Reinforcement Learning with Neural Reward Functions” on OpenResearch

About this paper

Robert Meier, Asier Mujika

This research from ETH Zürich introduces an open-ended reinforcement learning algorithm that leverages neural reward functions to iteratively discover complex skills. The method enables agents to learn a diverse repertoire of behaviors in high-dimensional and pixel-based environments without requiring manual feature engineering, achieving performance comparable to supervised agents in zero-shot transfer evaluations.

Key insights

• Neural networks can serve as effective, learnable reward functions, enabling an iterative, open-ended curriculum that progresses an agent through increasingly complex skill discovery without human intervention.
• Effective knowledge transfer mechanisms are crucial for mitigating the computational cost of learning new policies in an iterative open-ended setting, allowing the agent to build upon prior capabilities.
• Dynamically adjusted exploration via adaptive entropy regularization is essential for navigating sparse reward landscapes and generating diverse states needed to evolve reward functions towards complex behaviors.

Summary by alphaXiv.