Reproduce “Lessons from Learning to Spin "Pens"” on OpenResearch

About this paper

Jun Wang, Ying Yuan, Haichuan Che, Haozhi Qi, Yi Ma, Jitendra Malik, Xiaolong Wang

Researchers from UC San Diego, Carnegie Mellon University, and UC Berkeley demonstrated continuous pen spinning by a robotic hand using a learning-based system. Their approach trains an oracle policy in simulation and then fine-tunes a proprioceptive policy using limited real-world data obtained from replaying oracle trajectories, achieving robust generalization to various unseen pen-like objects with improved success rates compared to pure open-loop replay.

Key insights

• A carefully designed initial state distribution and a 'z-reward' encouraging horizontal object orientation are crucial for training stable oracle policies for dynamic manipulation in simulation.
• For highly dynamic and contact-rich tasks like continuous pen spinning, direct sim-to-real policy transfer or distillation is largely ineffective due to the substantial sim-to-real gap.
• Combining diverse motion priors from large-scale simulation pre-training with efficient real-world fine-tuning using limited, high-quality data is essential for robust generalization and cannot be simply replaced by more real-world data.

Summary by alphaXiv.