Reproduce “RIZE: Adaptive Regularization for Imitation Learning” on OpenResearch

About this paper

Adib Karimi, Mohammad Mehdi Ebadzadeh

Researchers at Amirkabir University of Technology developed RIZE, an Inverse Reinforcement Learning framework that uses adaptive regularization for implicit rewards and integrates distributional reinforcement learning. This approach achieved expert-level performance on complex continuous-control tasks like Humanoid-v2, often being the only method to solve it, and exhibited superior aggregate performance across various MuJoCo and Adroit environments.

Key insights

• Adaptive regularization allows for flexible, context-sensitive bounds on recovered implicit rewards, improving stability and performance compared to fixed targets.
• Integrating distributional value functions (IQN) provides a richer representation of return uncertainty, leading to more robust and sample-efficient policy learning in non-adversarial MaxEnt IRL.
• The combination of adaptive regularization and distributional critics yields bounded implicit rewards and Q-values, which contributes to more stable policy optimization and superior performance on challenging tasks.

Summary by alphaXiv.