Reproduce “Learning to Fill the Seam by Vision: Sub-millimeter Peg-in-hole on Unseen Shapes in Real World” on OpenResearch

About this paper

Liang Xie, Hongxiang Yu, Yinghao Zhao, Haodong Zhang, Zhongxiang Zhou, Minhang Wang, Yue Wang, Rong Xiong

The Seam Filling Net (SFN) framework enables robotic sub-millimeter precision peg-in-hole insertion for unseen shapes by leveraging a human-inspired "seam representation" and a multi-step reinforcement learning policy. This approach achieves high success rates with clearances down to 0.6mm and facilitates efficient, safe sim-to-real transfer through a decoupled perception and control architecture.

Key insights

• The "seam representation" provides a strong inductive bias that significantly enhances the generalization capabilities of vision-guided policies to novel geometries.
• Decoupling perception from policy through a semantic intermediate representation enables a highly efficient and safe sim-to-real transfer by localizing real-world adaptation to a single, easily retrainable module.
• Multi-step, multi-frame decision-making with reinforcement learning substantially improves the robustness of peg insertion against large initial errors and vision occlusion compared to step-optimal policies.

Summary by alphaXiv.