Reproduce “Cross-Embodiment Offline Reinforcement Learning for Heterogeneous Robot Datasets” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Haruki Abe, Takayuki Osa, Yusuke Mukuta, Tatsuya Harada

Researchers from The University of Tokyo and RIKEN developed a framework for cross-embodiment offline reinforcement learning that effectively utilizes heterogeneous robot datasets, including suboptimal data. Their Embodiment Grouping strategy, which clusters robots by morphological similarity for policy updates, mitigated inter-robot gradient conflicts, leading to a 33.99% average performance improvement on suboptimal datasets compared to standard cross-embodiment offline RL.

Key insights

  • Offline reinforcement learning (IQL) consistently outperforms Behavior Cloning (BC) when leveraging suboptimal robot interaction data in cross-embodiment settings.
  • Cross-embodiment pre-training significantly accelerates the fine-tuning process for individual robots, indicating positive transfer of general control capabilities.
  • Negative transfer observed in cross-embodiment learning, especially with high proportions of suboptimal data and diverse robots, stems from conflicting policy gradients among different robot morphologies.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.