Reproduce “Adversarial Imitation via Variational Inverse Reinforcement Learning” on OpenResearch

About this paper

Ahmed H. Qureshi, Byron Boots, Michael C. Yip

This research introduces Empowerment-regularized Adversarial Inverse Reinforcement Learning (EAIRL), a framework that learns robust, state-action dependent reward functions and generalizable policies from expert demonstrations. EAIRL integrates an empowerment-based regularization term, improving reward transferability and policy performance in complex control tasks.

Key insights

• Empowerment regularization guides the agent to explore states where it has greater influence over its environment, preventing premature convergence to local optima and promoting more robust reward learning.
• Modeling reward functions as dependent on both state and action is crucial for accurately recovering underlying reward structures, particularly in tasks involving action-dependent penalties (e.g., torque limits).
• An integrated adversarial framework can effectively leverage maximum entropy IRL principles with empowerment to learn highly transferable reward functions and generalizable policies simultaneously.

Summary by alphaXiv.