Reproduce “Sequences of Logits Reveal the Low Rank Structure of Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Noah Golowich, Allen Liu, Abhishek Shetty

Modern Large Language Models consistently demonstrate a low-rank structure in their extended logit matrices, an observation that allows for generating coherent text by linearly combining outputs from unrelated prompts and establishes a new architecture-agnostic theoretical framework for understanding these models.

Key insights

  • LLMs inherently exhibit a low-rank structure in their sequential output logits, implying intrinsic compressibility in this logit space.
  • This low-rank property emerges and refines during pre-training, rather than being present at initialization.
  • Linear relationships between histories captured by the low-rank structure are robust, largely invariant to specific future contexts (even nonsensical ones) and across different LLMs.
  • A language model possesses low logit rank if and only if it can be expressed as a time-varying ISAN, providing a tractable theoretical model for this observed property.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.