Reproduce “NeuronScope: A Multi-Agent Framework for Explaining Polysemantic Neurons in Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Weiqi Liu, Yongliang Miao, Haiyan Zhao, Yanguang Liu, Mengnan Du
NeuronScope explains polysemantic neurons by separating broad explanations into semantic conditions, grouping related conditions, and refining them against activation behavior. In a controlled comparison on Gemma-2-2B layer 9, adding refinement raised simulator-based held-out activation prediction Score from 0.539 to 0.620, while Number remained 2.49. The result supports improved simulator alignment in tested settings, not human-validated or complete interpretations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.