Reproduce “CoinRobot: Generalized End-to-end Robotic Learning for Physical Intelligence” on OpenResearch

About this paper

Yu Zhao, Huxian Liu, Xiang Chen, Jiankai Sun, Jiahuan Yan, Luhui Hu

Researchers from ZhiCheng AI, Peking University, and Stanford University developed CoinRobot, a generalized end-to-end robotic learning framework that leverages diffusion-based policies and a modular software system to achieve cross-platform adaptability. The framework demonstrated superior generalizability and robustness across diverse physical manipulation tasks and environmental conditions, with the complete dataset and trained models released to the community.

Key insights

• Diffusion-based policies within the CoinRobot framework consistently offer superior performance and generalizability, particularly for tasks requiring diverse actions and varied item locations, compared to explicit policies like ACT.
• A robust perception encoder, specifically an FPN-based ResNet34, is crucial for effectively capturing multi-resolution visual features and significantly enhancing overall model performance.
• Multi-task learning strategies and fine-tuning on combined datasets enable a single model checkpoint to generalize efficiently across distinct tasks and demonstrate robustness to unseen camera configurations.

Summary by alphaXiv.