Reproduce “Contextual Credit Policy Optimization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Huiwen Han, Ziyao Zeng, Zihao Zhu, Sisong Bei, Hao Wang, Hongyu Xu, Yan Han, Degui Zhi and 1 more
CCPO trains language-model agents to assign step-level credit in long-horizon tasks. It compares decisions from the same task and observation, weighting peers from other trajectories by contextual similarity, and combines realized returns with changes in contextual value over short continuations. On Qwen2.5-7B-Instruct, it reaches 97.4% on ALFWorld and 79.4% on WebShop, versus GRPO’s 82.0% and 72.9%; results cover two benchmarks and Qwen2.5-Instruct backbones.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.