Reproduce “VR-DAgger: Immersive VR for Dexterous Data Collection and Uncertainty-Guided On-Policy Correction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
René Zurbrügg, Tifanny Portela, Arjun Bhardwaj, Aravind Elanjimattathil Vijayan, Maximum Wilder-Smith, Marco Hutter
Researchers at ETH Zurich developed VR-DAgger, a human-in-the-loop framework that enhances robotic manipulation policy performance by efficiently collecting high-quality data. It leverages immersive VR for dexterous teleoperation and an uncertainty-guided mechanism to target and correct informative failure segments from policy rollouts, improving task success rates and generalization while reducing human supervision effort by around 40%.
Key insights
- On-policy iterative data collection, particularly through human-in-the-loop correction of failures, substantially improves policy success rates and generalization to varied initial conditions compared to offline behavioral cloning.
- An uncertainty-guided approach, identifying critical failure segments using Monte Carlo dropout, efficiently directs human supervision to the most informative states, reducing the need for continuous oversight.
- Immersive VR offers an intuitive and efficient interface for dexterous teleoperation, enabling high-quality demonstration and targeted corrective action during policy refinement.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.