Reproduce “Solver-Like Behavior in Large Language Models as Formalizers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Rikhil Amonkar, Stuti Mohan, Kevin Nguyen, Ceyhun Efe Kayan, Ronan Le Bras, Li Zhang
The authors measure solver-like behavior (SLB)—reasoning that assigns, tests, or rules out unknown values—in 2,000 generations from three models across constraint-satisfaction datasets. SLB appeared in 85% of traces and comprised 24% of reasoning tokens. Hiding instance values nearly eliminated SLB without detected accuracy loss in tested settings; results do not establish lower token use or generalize beyond them.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.