Reproduce “Sparse Readout Prism: Explaining Logit-Lens Scores in Features Instead of Tokens” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Matteo He, William F. Shen, Xinchi Qiu, Nicholas D. Lane

Sparse Readout Prism (SRP) decomposes language-model readout weights into reusable sparse features, explaining lens-derived token scores through feature contributions and residuals rather than token identity alone. On Qwen3.5-2B selected logit differences, SRP coverage was 0.780 versus 0.625 for the strongest row-geometry baseline. In fitted-lens studies, surface tokens changed while dominant features usually did not; this does not establish internal language.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.