Reproduce “The First Impression Problem: Internal Bias Triggers Overthinking in Reasoning Models” on OpenResearch

About this paper

Renfei Dang, Zhening Li, Shujian Huang, Jiajun Chen

Researchers at Nanjing University identified "internal bias," a model's preliminary intuitive guess, as a fundamental trigger for "overthinking" in R1-type reasoning models. A conflict between this initial bias and subsequent deliberate reasoning was found to increase reasoning length by 21.0% to 43.1%, and an attention-based early-exit mechanism showed promise in mitigating this effect.

Key insights

• Reasoning models consistently form an early "internal bias" (an intuitive answer guess) that is often incorrect, and its deviation from the final answer strongly correlates with increased reasoning length.
• The model's attention mechanisms exhibit excessive focus on the input question at "reflection points," reactivating internal biases and perpetuating redundant reasoning loops.
• Existing overthinking mitigation methods primarily reduce average reasoning length but often fail to eliminate the *influence* of internal bias, indicating its resilience and the need for targeted solutions.

Summary by alphaXiv.