Reproduce “Function-Preserving Data Generation for Zero-Shot Real-to-Sim-to-Real Manipulation” on OpenResearch

About this paper

Tianyi Xiang, Xupeng Xie, Jiahang Cao, Andrew F. Luo, Haoang Li, Jun Ma

Function-Preserving Shape Augmentation expands a reconstructed real-world manipulation scene into a broader simulation training distribution while keeping task-critical interfaces, poses, and collision relationships valid. Combined with visual and calibration randomization, the approach trains Diffusion Policies without real demonstrations or fine-tuning, reaching 82% average nominal success across five real-robot tasks and outperforming unconstrained scaling and shape-library baselines on contact-rich manipulation.

Key insights

• Geometric diversity is most useful when it preserves the physical interfaces that determine grasping, insertion, fastening, assembly, and extraction success; visual plausibility alone is insufficient.
• Under a fixed budget of 3,000 episodes, spreading data across more valid shapes improves simulation performance, especially for contact-rich and long-horizon tasks, whereas simple pick-up performance saturates earlier.
• Real-to-sim reconstruction is treated as a prior for generating a category-level training distribution rather than as a single fixed environment, avoiding the need for separate teleoperated trajectories for every object shape.

Summary by alphaXiv.