Reproduce “Visual Reinforcement Learning with Self-Supervised 3D Representations” on OpenResearch

About this paper

Yanjie Ze, Nicklas Hansen, Yinbo Chen, Mohit Jain, Xiaolong Wang

A framework for visual reinforcement learning leverages self-supervised 3D representations to improve sample efficiency and achieve superior zero-shot sim-to-real transfer for robotic manipulation tasks. This approach enables higher success rates on physical hardware, such as 46% on the 'lift' task compared to 20% for 2D baselines, while also enhancing robustness to visual perturbations.

Key insights

• Explicitly incorporating 3D geometry into visual representations provides stronger inductive biases for RL, leading to improved sample efficiency and generalization in robotic manipulation tasks.
• Jointly finetuning the self-supervised 3D representation alongside the RL policy yields superior performance and robustness compared to using frozen pretrained 2D or 3D features.
• Pretraining on large-scale object-centric 3D datasets is essential for establishing foundational geometric understanding, which can then be refined with task-specific data.

Summary by alphaXiv.