Reproduce “Sim2Real Manipulation on Unknown Objects with Tactile-based Reinforcement Learning” on OpenResearch

About this paper

Entong Su, Chengzhe Jia, Yuzhe Qin, Wenxuan Zhou, Annabella Macaluso, Binghao Huang, Xiaolong Wang

Researchers from UCSD, CMU, and UIUC developed a reinforcement learning policy for robot manipulation that learns to pivot unknown objects using tactile feedback, entirely in simulation. Their method, employing simplified binary tactile image representations and diverse object training, achieves zero-shot transfer to a real robot with an 80% success rate on unseen objects and surfaces.

Key insights

• Simplified tactile representations, particularly binary images, are more effective for robust Sim2Real transfer than raw RGB data, as they abstract away non-essential visual details and reduce sensitivity to real-world variations.
• Training with a large diversity of objects in simulation is crucial for achieving strong generalization capabilities when deploying policies on unseen and unknown objects in the real world.
• End-to-end reinforcement learning with tactile input is more robust than methods relying on explicit object pose or geometry estimation from tactile data, which are prone to real-world noise and inaccuracies.

Summary by alphaXiv.