Reproduce “Integrating Model-based Control and RL for Sim2Real Transfer of Tight Insertion Policies” on OpenResearch

About this paper

Isidoros Marougkas, Dhruv Metha Ramesh, Joe H. Doerr, Edgar Granados, Aravind Sivaramakrishnan, Abdeslam Boularias, Kostas E. Bekris

Researchers at Rutgers University developed a hybrid control framework for robotic tight insertion tasks, integrating a potential field-based controller with a residual reinforcement learning policy. This approach enables zero-shot sim-to-real transfer, achieving success rates exceeding 90% in real-world experiments on diverse objects without requiring real-world fine-tuning.

Key insights

• Combining a model-based potential field controller with residual reinforcement learning enables robust sim-to-real transfer for precise robotic manipulation tasks by learning only the necessary corrections.
• A multi-dimensional curriculum learning strategy, varying observation noise and action magnitude, effectively trains policies to handle real-world uncertainties without explicit real-world data.
• The model-based component provides a stable foundation, allowing the reinforcement learning policy to focus on fine adjustments, which significantly improves training efficiency and generalization.

Summary by alphaXiv.