Reproduce “Excavation Reinforcement Learning Using Geometric Representation” on OpenResearch

About this paper

Qingkai Lu, Yifan Zhu, Liangjun Zhang

A self-supervised geometric representation for point clouds enables reinforcement learning to perform sequential excavation of irregular rigid objects in cluttered environments. This approach improves RL training efficiency and achieves more robust sim2real transfer, notably reducing planning failures on a physical robot compared to end-to-end learning.

Key insights

• Learning a self-supervised geometric representation from point clouds, without human labels, provides a compact and effective state input that drastically improves DRL training efficiency and sim2real transfer for complex manipulation.
• This geometric representation enhances policy robustness by enabling the RL agent to learn actions that avoid physical constraints (e.g., IK failures, self-collisions) more reliably in both simulated and real environments.
• Reinforcement learning can successfully tackle sequential excavation of irregular rigid objects in cluttered scenes, a challenging problem often intractable for traditional model-based approaches, when leveraging suitable geometric scene understanding.

Summary by alphaXiv.