Reproduce “Real2Gym: Building Gyms from Videos, Bringing Skills to Robots” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kerui Ren, Yingxiang Xu, Kaiwen Song, Linning Xu, Bo Dai, Mulin Yu, Tao Lu
Real2Gym turns a demonstration into a physics-interactive digital twin, where an agent practices manipulation, extracts object-relative skills, and grounds them in robot observations. Across 12 DROID and 12 EgoDex tasks, skill-conditioned policies achieved 87.5% success versus 70.8% for GPT-6 Astra Direct Mode, using 0.50 million versus 1.99 million tokens. Real-world transfer was demonstrated on one plate-insertion task.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.