Reproduce “IIFL: Implicit Interactive Fleet Learning from Heterogeneous Human Supervisors” on OpenResearch

About this paper

Gaurav Datta, Ryan Hoque, Anrui Gu, Eugen Solowjow, Ken Goldberg

The IIFL algorithm enhances robot learning within a fleet by integrating interactive learning to combat distribution shift with implicit policies to handle multimodal human demonstrations. It achieves up to 2.8x higher success rates and 3.4x greater Return on Human Effort compared to explicit Interactive Fleet Learning in heterogeneous control simulations, and an 80% improvement in successes over Implicit Behavior Cloning in physical robot tasks.

Key insights

• Combining interactive learning for distribution shift robustness with implicit policies for multimodal action representation is crucial for effective robot learning from diverse human demonstrations.
• Jeffreys divergence offers a computationally tractable and effective measure of epistemic uncertainty for Energy-Based Models, which is vital for robot-gated interactive learning in EBM-based systems.
• Intelligently allocating human supervisors based on robot uncertainty significantly enhances the efficiency and autonomy of fleet learning from heterogeneous human input.
• The benefits of implicit policies extend beyond multimodality, potentially offering advantages over explicit policies even when demonstrations are seemingly unimodal.

Summary by alphaXiv.