Reproduce “Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro and 18 more
The authors train sparse autoencoders (SAEs) on Claude 3 Sonnet’s middle-layer activations to identify features and test them by intervening on feature values. The dictionaries contain up to 34 million features, though coverage is incomplete. Raising a code-error feature makes Claude report errors in correct code, while suppressing it can hide bugs; this demonstrates steerability, not normal behavior or a safety control.
Key insights
- Dictionary learning with SAEs is a scalable and effective approach for extracting monosemantic, interpretable features from production-grade large language models.
- Features in Claude 3 Sonnet are highly abstract, multilingual, multimodal, and generalize across concrete and abstract instantiations of concepts, indicating sophisticated internal representations.
- SAE features offer significantly greater interpretability and specificity compared to individual neurons, effectively unpacking superposition into distinct, meaningful computational components.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.