Reproduce “Squint: Fast Visual Reinforcement Learning for Sim-to-Real Robotics” on OpenResearch

About this paper

Abdulaziz Almuzairee, Henrik I. Christensen

Squint trains visual robot-manipulation policies by reusing experience from parallel simulations while reducing image-processing costs. It renders images at 128 × 128 and area-downsamples them to 16 × 16, alongside an optimized Soft Actor-Critic system. On eight real-world tasks, it succeeds in 73 of 80 trials (91.3%), but Stack Can succeeds in 6 of 10.

Key insights

• Training throughput and experience reuse can be combined: Squint retains off-policy replay while using parallel simulation and implementation optimizations to reduce elapsed training time.
• The tested configuration’s performance comes from interacting design choices, including downsampling rendered images, layer normalization, a higher update-to-data ratio, and a large replay buffer.
• Visual randomization matters for transfer: removing color jitter lowered real-robot success from 91.3% to 72.5%, while larger 32×32 inputs matched transfer success but took longer to train.

Summary by alphaXiv.