Reproduce “Imitation Learning by Estimating Expertise of Demonstrators” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mark Beliaev, Andy Shih, Stefano Ermon, Dorsa Sadigh, Ramtin Pedarsani
ILEED improves imitation learning from heterogeneous demonstrations by jointly learning a policy and state-dependent expertise scores for each demonstrator, using only state-action data and demonstrator identity. Across simulated navigation, robotic manipulation, chess, and complementary-skill tasks, it generally outperformed behavioral cloning and could combine useful abilities from otherwise imperfect demonstrators.
Key insights
- Demonstrator identity can be treated as structured information rather than simply as a source of noisy labels, enabling the learner to discount unreliable actions selectively.
- State-dependent expertise is important for combining complementary skills: different demonstrators can be trusted in different regions of the task rather than assigned one global quality score.
- The likelihood formulation provides a principled training and model-selection objective, but reliable policy recovery depends on assumptions about model specification, representation, and identifiability.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.