Reproduce “LATTS: Locally Adaptive Test-Time Scaling” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Theo Uscidda, Matthew Trager, Michael Kleinman, Aditya Chattopadhyay, Wei Xia, Stefano Soatto

LATTS (Locally Adaptive Test-Time Scaling) dynamically adjusts computational effort at each step of a Chain-of-Thought reasoning process for Large Language Models. It demonstrates improved accuracy-compute tradeoffs on mathematical reasoning benchmarks, allowing smaller base models to achieve performance comparable to or exceeding substantially larger models while consuming considerably fewer tokens.

Key insights

  • The method implicitly scales computation: "Difficult" reasoning steps, where the base model struggles to generate high-scoring candidates, naturally require more sampling attempts and thus more compute.
  • LATTS can be conceptualized as a form of speculative decoding adapted for multi-step free-text reasoning, where the base LLM acts as the draft model and verification occurs at the step level.
  • The choice of verifier modulator (e.g., identity vs. hard threshold) and fallback strategy significantly impacts the accuracy-compute tradeoff, with different configurations optimal for varying compute budgets and problem types.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.