Reproduce “CalibForge: Adversarial Solver Calibration for Scaling Learnable Terminal Tasks” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Fanzhe Meng, Guoxin Chen, Jiale Zhao, Shuang Sun, Yuzhi Lin, Wayne Xin Zhao, Ruihua Song, Ji-Rong Wen and 1 more

CalibForge trains terminal agents using tasks revised through solver attempts. It probes candidate tasks with independent solvers, then uses verified pass/fail outcomes and attempt traces to revise and retain tasks that meet a specified solver-relative criterion. In a matched ablation, multi-solver calibration reached 29.21% ± 1.12 accuracy, versus 23.97% ± 1.98 with single-solver feedback on Terminal-Bench 2.1.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.