Reproduce “As Language Models Scale, Low-order Linear Depth Dynamics Emerge” on OpenResearch

About this paper

Buddhika Nettasinghe, Geethu Joseph

This research demonstrates that the depth dynamics of large language models, specifically the propagation of the last-token hidden state, can be accurately approximated by low-order linear dynamical systems within specific contexts. This approach yields precise predictions of layerwise steering sensitivity and enables the design of minimal-energy multi-layer control policies that outperform heuristic methods.

Key insights

• The fidelity and identifiability of low-order linear depth dynamics surrogates improve monotonically with increasing model scale (e.g., from GPT-2 to GPT-2-large).
• The emergent linear dynamics provide a tractable mechanistic abstraction, offering accurate prediction of internal information flow and challenging the traditional "black box" view of LLMs.
• Principled multi-layer intervention policies designed using the linear surrogate achieve desired effects with significantly less total injection energy compared to heuristic methods.

Summary by alphaXiv.