Reproduce “How to Utilize Failure Demo Data?: Effective Data Selection for Imitation Learning Using Distribution Differences in Attention Mechanism” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Kana Miyamoto, Kanata Suzuki, Tetsuya Ogata

Researchers from Waseda University and Fujitsu Limited developed a method for imitation learning that effectively utilizes failed robot demonstrations to improve task performance. Their approach, which employs learnable parametric bias and KL regularization within a transformer model, achieves an increase in success rate from 71.6% to 77.8% in a simulated cube-lifting task and up to 79.4% with selective use of 'near-success' failure data.

Key insights

  • Attention mechanisms within imitation learning models can learn and represent distinct patterns that differentiate successful from failed robot trajectories.
  • Not all failure data is equally useful; failures with smaller attention discrepancies from successful demonstrations ('near-success' cases) are most informative for improving policy learning.
  • A quantitative metric based on attention distribution differences can effectively identify beneficial failure samples for targeted retraining.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.