Reproduce “Grounding Sim-to-Real Generalization in Robotic Manipulation: An Empirical Study with Vision-Language-Action Models” on OpenResearch

About this paper

Ruixing Jin, Zicheng Zhu, Ruixiang Ouyang, Sheng Xu, Bo Yue, Zhizheng Wu, Guiliang Liu

This empirical study investigates factors influencing Sim-to-Real generalization for Vision-Language-Action (VLA) models in robotic manipulation, identifying spatial randomization and frame-wise variations as critical drivers for robust transfer, with reinforcement learning fine-tuning providing further resilience to real-world distribution shifts. Researchers evaluated various techniques through over 10,000 real-world trials and released a standardized benchmark platform.

Key insights

• Spatial randomization (e.g., camera pose, table height) is the primary determinant of zero-shot Sim2Real generalization for VLA models, with appearance randomization providing complementary benefits.
• Frame-wise application of randomization, which introduces continuous variations within an episode, consistently outperforms episode-wise randomization by preventing policies from overfitting to static scene configurations.
• Reinforcement learning fine-tuning significantly enhances policy robustness to real-world distribution shifts, exhibiting synergistic performance gains when combined with comprehensive domain randomization.

Summary by alphaXiv.