Reproduce “Entropy-Lens: Uncovering Decision Strategies in LLMs” on OpenResearch

About this paper

Riccardo Ali, Francesco Caso, Christopher Irwin, Pietro Liò

Researchers at the University of Cambridge developed Entropy-Lens, a framework that applies information-theoretic entropy to logit-lens predictions, providing a scalar, layer-by-layer view of how Large Language Models refine next-token predictions. This approach mechanistically linked entropy changes to either an "expansion" or "pruning" of candidate tokens, revealing family-specific, depth-invariant, and task/format-dependent computational signatures, and demonstrating that expansion phases are generally more critical for model accuracy.

Key insights

• Changes in intermediate logit entropy monotonically correlate with the expansion or pruning of candidate tokens, providing a mechanistic interpretation of LLM decision-making.
• LLMs exhibit characteristic and consistent entropy profiles that are distinct for different model families and tasks, demonstrating depth-rescaling invariance within families.
• The identified expansion and pruning phases have unequal functional importance, with expansion often being more critical for downstream task performance, as shown by targeted intervention experiments.

Summary by alphaXiv.