Reproduce “Visual Sim-to-Real Learning for Robotic Insertion under Geometric Variations: Application to Rebar Installation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tao Sun, Beining Han, Patrick Yin, Rui Xu, Harry He, Abhishek Gupta, Szymon Rusinkiewicz, Yi Shao
RebarSim generates rebar geometries in simulation, trains a privileged reinforcement-learning teacher, and distills its actions into a camera-guided student using eight RGB views and proprioception. The frozen student inserts already-grasped bars without demonstrations. Success requires all three sections within 5 mm; it achieved 137 of 150 seatings, or 91.3%, on two training-grid designs from production batches, not out-of-grid tests.
Key insights
- Varying nominal rebar designs provides more transferable geometric knowledge than varying fabrication tolerances for only one design; combining both sources of diversity produces the strongest results.
- Pretraining on diverse designs improves adaptation to extrapolated geometries: the teacher reaches 92.6% zero-shot success, while the student reaches 85% after about 0.19 million distillation samples.
- Appearance randomization is essential under visual distribution shifts, and DAgger substantially outperforms pure behavior cloning by exposing the policy to student-induced failure states.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.