Reproduce “Learning Causally Invariant Reward Functions from Diverse Demonstrations” on OpenResearch

About this paper

Ivan Ovinnikov, Eugene Bykovets, Joachim M. Buhmann

Researchers at ETH Zürich developed a causal invariance (CI) regularization method for Inverse Reinforcement Learning (IRL) to learn reward functions that are robust to spurious correlations. This approach enabled policies trained on the recovered rewards to generalize more effectively under environmental dynamics shifts in both gridworld and complex robotic tasks.

Key insights

• Variations within diverse expert demonstrations can be effectively modeled as interventional settings where the underlying conditional distribution of optimality for a given transition should remain invariant.
• Applying causal invariance as a regularization principle effectively mitigates 'behavioral overfitting' by encouraging reward functions to focus on causally stable features, even when expert data is limited or heterogeneous.
• The proposed method offers a principled way to extract a core, generalizable task goal from expert behaviors, independent of superficial variations or preferences.

Summary by alphaXiv.