Reproduce “AR2-D2:Training a Robot Without a Robot” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiafei Duan, Yi Ru Wang, Mohit Shridhar, Dieter Fox, Ranjay Krishna
AR2-D2 is an iOS application that enables users to generate robot manipulation demonstrations using augmented reality on a mobile device, eliminating the need for physical robots or specialized training. Policies trained with this AR-generated data achieved comparable task success rates to those trained with real-robot demonstrations, while significantly reducing data collection time.
Key insights
- Augmented reality on ubiquitous mobile devices can effectively democratize robot data collection, allowing a broader user base to generate high-quality demonstrations for personalized object manipulation.
- Collecting robot demonstrations in AR significantly reduces data collection time and user training requirements compared to traditional methods while maintaining comparable task success rates.
- Policies trained primarily with AR-generated data can successfully transfer to and operate on real robots manipulating personalized objects, with minimal fine-tuning needed to bridge the sim-to-real domain gap.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.