Reproduce “Actor, Judge, or Teacher? Understanding Decision Models in Long-Horizon Agents” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Haochen Liu, Qianhao Zhu, Zixuan Wang, Dingming Li, Tongbo Chen, Peng Qiu, Yingqi Qu, Weiming Lu and 2 more

JEV uses one fixed decision model as Actor, Judge, and Teacher: it scores rollout actions after the trajectory, and centered scores augment GRPO outcome advantages during training. For Qwen2.5-1.5B-Instruct, Hindsight + Progress raises ALFWorld All success from 72.8% ± 3.6 to 82.9% ± 2.9, and WebShop success from 56.8% ± 3.8 to 71.4% ± 0.7, in two text environments with episodes capped at 30 and 15 actions, respectively.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.