Reproduce “Causal Imitation Learning under Expert-Observable and Expert-Unobservable Confounding” on OpenResearch

About this paper

Daqian Shao, Thomas Kleine Buening, Marta Kwiatkowska

Researchers from Oxford and ETH Zurich developed a causal imitation learning framework that addresses both expert-observable and expert-unobservable confounding variables by reformulating the problem as a Conditional Moment Restriction. Their DML-IL algorithm consistently outperformed existing baselines in various custom and MuJoCo environments, achieving lower Mean Squared Error and higher average rewards by effectively accounting for these hidden factors.

Key insights

• Trajectory histories can effectively serve as instrumental variables to identify the expert's causal policy, even when both expert-observable and expert-unobservable confounders are present.
• Reformulating causal IL as a Conditional Moment Restriction problem enables the application of robust econometric tools and provides a principled approach for developing algorithms with theoretical guarantees.
• The imitation gap is influenced by three key factors: the imitator's ability to infer expert-observable confounders from history, the strength of the instrumental variable (confounding horizon), and the disturbance caused by expert-unobservable confounding noise.

Summary by alphaXiv.