Reproduce “Understanding How Value Neurons Shape the Generation of Specified Values in LLMs” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yi Su, Jiayi Zhang, Shu Yang, Xinhai Wang, Lijie Hu, Di Wang

Researchers from King Abdullah University of Science and Technology and the University of Copenhagen developed ValueLocate, a framework that leverages a new dataset (ValueInsight) to identify and causally manipulate specific neurons associated with human values in Large Language Models. Their method effectively modifies LLMs' value orientations and reveals that less than 0.4% of neurons, concentrated in middle layers, influence these values.

Key insights

  • Value orientations in LLMs are influenced by a sparse subset of specialized neurons, accounting for less than 0.4% of total neurons per layer.
  • These value-related neurons are consistently concentrated in the middle layers of LLMs (e.g., around the 15th layer in LLama-3.1-8B), suggesting a structured organizational pattern for value encoding.
  • A direct causal link exists between the identified value-related neurons and an LLM's expressed value representations, enabling targeted manipulation.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.