Reproduce “Discovering Millions of Interpretable Features with Sparse Autoencoders” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
XinYang He, Wei Wang, Bing Zhao, Xuan Ren, WenBo Li, WeiXu Qiao, Hu Wei, Lin Qu
Researchers from Alibaba Group Holding Limited release Qwen3-Instruct SAE, a comprehensive collection of Sparse Autoencoders for instruction-tuned Qwen3 models across various scales and activation sites. This resource enables the decomposition of dense internal representations into interpretable features and demonstrates the ability to causally steer model behavior, such as inducing refusal, by intervening on specific SAE features.
Key insights
- Sparse Autoencoders effectively decompose superposed representations into interpretable features in instruction-tuned models, with reconstruction fidelity and model recovery varying by activation site and layer.
- Residual stream and MLP output representations generally exhibit better model performance recovery than attention output representations, indicating their direct encoding of semantic content.
- Specific SAE features can be causally manipulated to steer instruction-tuned LLM behavior, demonstrating fine-grained control over model outputs beyond prompt engineering.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.