Reproduce “Long-Context Generation Is a Sampling Problem (When You Turn Up the Heat)” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Allen G. Roush, Minh Nhat Nguyen, Ravid Shwartz-Ziv, Judah Goldfeder, Sanjay Basu

The paper tests sampling rules for high-temperature long-form generation and coding agents. Top-nσ keeps tokens within a chosen number of standard deviations of the highest logit, making its candidate set invariant to temperature for fixed logits. At temperature 2, it resolves 51 of 100 SWE-bench Verified issues on Qwen3.5-9B, versus zero for top-p; this result is limited to that model and issue set.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.