Reproduce “On Turn-Level Best-of-N Sampling for Scaling LLM Agents” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jalaj Bhandari, Dhawal Gupta, Hongli Zhan, Zhenting Wang, Varad Pimpalkhute, Zhengzhong Liu, Eric Xing, Rupesh Srivastava

The authors introduce Turn-BoN, a terminal-agent method that samples candidate actions at each turn and uses an off-the-shelf language model to select one before execution. Only the selected action changes the environment, whose feedback informs the next choice. On Terminal-Bench v2.1, Qwen3.6-35B’s pass rate rises from 47.5% to 55.2% at N=8, pooling two selectors, at higher inference-time output-token cost.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.