Reproduce “An Unbiased Look at Datasets for Visuo-Motor Pre-Training” on OpenResearch

About this paper

Sudeep Dasari, Mohan Kumar Srirama, Unnat Jain, Abhinav Gupta

An empirical study investigates the impact of pre-training dataset choice on visuo-motor learning for robotic manipulation. The research demonstrates that traditional computer vision datasets often yield stronger real-world performance, and certain dataset combinations can boost success rates by 30% over state-of-the-art baselines.

Key insights

• Traditional vision datasets (ImageNet, Kinetics, 100 Days of Hands) generally produce more effective visuo-motor representations for real-world robotics than specialized egocentric (Ego4D) or robot-specific (RoboNet) data.
• The distribution and diversity of images within a pre-training dataset are more critical for downstream task performance than simply increasing the raw volume of data.
• Simulation benchmarks exhibit a weak correlation with real-world robotic task success, rendering them unreliable predictors for evaluating pre-training efficacy.

Summary by alphaXiv.