Reproduce “Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning” on OpenResearch

About this paper

Andy Zeng, Shuran Song, Stefan Welker, Johnny Lee, Alberto Rodriguez, Thomas Funkhouser

A self-supervised deep reinforcement learning framework enables a robot to learn synergistic pushing and grasping policies directly from visual input. This approach significantly boosts grasping success and efficiency in cluttered environments by autonomously discovering complex manipulation strategies through joint action-value functions.

Key insights

• Complex synergies between pushing and grasping can be learned end-to-end from scratch, directly from visual observations, without explicit object models or hand-engineered features.
• Pushing actions are effectively learned when their reward is primarily derived from their contribution to enabling future successful grasps, demonstrating a discoverable long-term utility.
• Fully convolutional networks are highly efficient for this task, allowing a single forward pass to compute Q-values for a large action space, making real-world training tractable within hours.

Summary by alphaXiv.