Reproduce “Towards Spectroscopy: Susceptibility Clusters in Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Andrew Gordon, Garrett Baker, George Wang, William Snell, Stan van Wingerden, Daniel Murfet
Researchers from Timaeus developed a novel interpretability framework, 'susceptibility spectroscopy,' that probes language models with data perturbations to infer their internal computational structures. This method identified 510 interpretable clusters in Pythia-14M, representing specialized responses to data patterns, and these structures showed 50.8% overlap with Sparse Autoencoder features, persisting in models up to 1.4B parameters.
Key insights
- The geometry of susceptibility space reflects the computational role of tokens, grouping identical tokens distinctly based on context and abstracting over diverse tokens performing similar functions.
- Susceptibility analysis uncovers 'networks' of linked clusters, representing the model's coordinated processing of multi-token, structured phenomena like HTML markup or code blocks.
- The discovered susceptibility clusters largely overlap with features found by Sparse Autoencoders (SAEs), providing cross-method validation for the recovery of genuine internal structures.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.