Reproduce “Base Models Can Reason By Taking a Cue From Training Data” on OpenResearch

About this paper

Sophie L. Wang, Amil Dravid, Rulin Shao, Kevin Farhat, Sewon Min, Alexei A. Efros

The authors test whether short fixed openings can elicit reasoning from language models without changing their weights. On MATH-500, Olmo-3-7B pass@1 rises from 42% without a cue to 78% with “Okay,” versus 75% after reinforcement learning. In a controlled data-edit experiment, renaming “okay” to “chicken” makes “Chicken” effective, supporting cue effects in these tested settings, not all models.

Key insights

• In the tested Olmo and Qwen models, a one- or two-token opening can make a base model’s reasoning performance approach or match that of its RL counterpart; the effect varies across models and tasks.
• RL increased the probability of effective openings, with the largest base–RL distribution differences occurring near response onset. Supplying a cue before RL training also reduced the updates needed to reach comparable accuracy in the reported runs.
• Training-data edits can transfer or redirect a cue’s behavior, supporting a causal link between data associations and elicited outputs. Cue effects depend on token position and exact formatting, and can alter refusal behavior as well as reasoning.

Summary by alphaXiv.