Reproduce “Grammatical "grandmother neurons" are rare in LLMs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Linyang He, Nima Mesgarani
The authors introduce the Neuron Separability Index (NSI), a probe-free measure of whether one activation dimension distinguishes matched grammatical and ungrammatical sentences beyond a label-swapping null. In Qwen3-0.6B, 65 of 68 paradigms have no unit above descriptive NSI>2; each remaining paradigm has one. Across seven tested models, selectivity is sparse in controlled English contrasts and does not establish causal reliance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.