Reproduce “On the Sample Efficiency of Inverse Dynamics Models for Semi-Supervised Imitation Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Sacha Morin, Moonsub Byeon, Alexia Jolicoeur-Martineau, Sébastien Lachapelle

Morin et al. provide a comprehensive explanation for the enhanced sample efficiency of Inverse Dynamics Model (IDM)-based methods in semi-supervised imitation learning, demonstrating that the ground-truth IDM is frequently less complex and less stochastic than the expert policy. This theoretical understanding is supported by empirical evidence where IDM-based policies consistently outperformed Behavior Cloning across Procgen and robot manipulation tasks, and a refined LAPO+ algorithm showed improved performance over LAPO.

Key insights

  • The ground-truth Inverse Dynamics Model (IDM) is often fundamentally simpler and less complex to learn than the expert policy, requiring fewer data points for accurate generalization.
  • Observing both current and next states for IDM training significantly reduces the inherent stochasticity of the target action, leading to a more deterministic and easier-to-learn mapping compared to the expert policy.
  • The implicit "simplicity bias" of neural networks further enhances learning efficiency for the simpler IDM compared to the more intricate expert policy, even with high-capacity models.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.