Reproduce “Heterogeneous Agent Collaborative Reinforcement Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zhixia Zhang, Zixuan Huang, Gongxun Li, Huaiyang Wang, Chengyi Yuan, Xin Xia, Deqing Wang, Fuzhen Zhuang and 5 more

HACPO lets heterogeneous reasoning models share verifier-checked responses during training, while each remains independently deployable at inference. It adjusts cross-model updates for differences in capability and response distributions. Across six model outcomes averaged over seven math benchmarks, HACPO improves accuracy by about 3.6 percentage points over GSPO×2 with half the rollout cost; gains are averages, not wins on every task.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.