Reproduce “Data Quality in Imitation Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Suneel Belkhale, Yuchen Cui, Dorsa Sadigh
Researchers from Stanford University formalize data quality in imitation learning by identifying action divergence and transition diversity as key properties affecting distribution shift. Their study reveals that consistent expert actions are crucial, and an optimal level of environmental noise can mitigate the negative effects of noisy demonstrations, challenging the sole focus on maximizing state diversity.
Key insights
- Simply maximizing state diversity is not always beneficial and can be detrimental if it increases action divergence.
- Action consistency in expert demonstrations is crucial for effective policy learning, as inconsistent actions lead to poor learning.
- Transition diversity (system noise) has an optimal range, where some level improves performance by exposing the policy to a wider state range, but excessive noise can degrade learning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.