Reproduce “A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasoners” on OpenResearch

About this paper

Bowen Jiang, Yangxinyu Xie, Zhuoqun Hao, Xiaomeng Wang, Tanwi Mallick, Weijie J. Su, Camillo J. Taylor, Dan Roth

Researchers from the University of Pennsylvania and Argonne National Laboratory demonstrate that large language models' apparent reasoning capabilities are consistently undermined by 'token bias,' where irrelevant changes to input tokens predictably shift model output. This work provides a statistically rigorous framework to expose LLMs' reliance on superficial patterns rather than genuine logical understanding.

Key insights

• Large Language Models (LLMs) consistently rely on superficial token patterns and contextual cues rather than deep logical understanding for reasoning tasks.
• Popular prompting techniques like Chain-of-Thought (CoT) may elicit an imitation of reasoning behavior, suggesting LLMs exploit semantic shortcuts rather than true logical deduction.
• The observed token bias indicates brittle performance, raising concerns about LLMs' ability to generalize robustly to novel problem phrasings in real-world scenarios.

Summary by alphaXiv.