Reproduce “Learning Object Manipulation from Scratch via Contrastive Interaction” on OpenResearch

About this paper

Tongle Shen, Caleb Chuck, Fan Feng, Biwei Huang

Shen et al. introduce Interaction-Weighted Resampling (IWR) to enhance Contrastive Reinforcement Learning (CRL) for robot object manipulation tasks. The method improved average performance by 19.8% across simulation benchmarks and achieved a 140% higher success rate on a real-world robot air hockey task compared to previous CRL approaches, effectively learning robust interaction policies.

Key insights

• Object manipulation dynamics can be conceptualized as a piecewise-smooth Markov process where interactions cause shifts in dynamic modes, leading to piecewise nonlinear reachability structures that challenge standard CRL representations.
• Standard CRL's implicit linear-Gaussian temporal model for future occupancy struggles to represent these mode-conditioned dynamics, often smoothing over critical interaction boundaries.
• Local errors in estimating interaction-specific dynamic shifts are formally shown to be amplified through subsequent passive dynamics, severely impacting the accuracy of learned energy functions for goal-reaching.

Summary by alphaXiv.