Reproduce “DemoStart: Demonstration-led auto-curriculum applied to sim-to-real with multi-fingered robots” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Maria Bauza, Jose Enrique Chen, Valentin Dalibard, Nimrod Gileadi, Roland Hafner, Murilo F. Martins, Joss Moore, Rugile Pevceviciute and 8 more
DemoStart uses demonstrations to select training restart states rather than imitate their actions, then trains a policy in simulation and distills it into a camera-based policy for a real robot. For plug insertion, it reaches 99.6% success in simulation and 64% on the real robot, versus 2% for real-teleoperation behavior cloning; this transfer remains substantially below simulation performance.
Key insights
- Zero-Variance Filtering (ZVF) is crucial for effective sparse-reward learning in complex environments, identifying optimal learning opportunities by focusing on states where the policy sometimes succeeds and sometimes fails.
- Leveraging minimal, suboptimal demonstrations to seed an auto-curriculum, rather than for direct imitation, allows the RL agent to discover more efficient and robust manipulation strategies not present in the original demonstrations.
- Combining demonstration-led auto-curriculum with extensive domain randomization and policy distillation enables robust zero-shot sim-to-real transfer for high-DoF multi-fingered manipulation directly from visual and proprioceptive inputs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.