Reproduce “Causal Confusion in Imitation Learning” on OpenResearch

About this paper

Pim de Haan, Dinesh Jayaraman, Sergey Levine

This paper identifies and addresses 'causal misidentification' in imitation learning, a problem where policies learn non-causal correlations from demonstrations that lead to failures when deployed. It proposes a targeted interventional framework that combines graph-parameterized policy learning with efficient expert queries or policy execution to identify and rely on true causal factors, resulting in significantly improved robustness and sample efficiency.

Key insights

• Behavioral cloning's discriminative nature makes it susceptible to learning non-causal correlations (causal misidentification) from expert demonstrations.
• This causal misidentification, exacerbated by distributional shift during policy deployment, leads to compounding errors and poor performance.
• Targeted active interventions, where the agent interacts with the environment or queries an expert, can efficiently identify the true causal factors needed for robust imitation.

Summary by alphaXiv.