Reproduce “Beyond Single Concept Vector: Modeling Concept Subspace in LLMs with Gaussian Distribution” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Haiyan Zhao, Heng Zhao, Bo Shen, Ali Payani, Fan Yang, Mengnan Du
The Gaussian Concept Subspace (GCS) framework models semantic concepts within Large Language Models (LLMs) as Gaussian distributions instead of single vectors, providing a more robust and consistent representation. This approach yielded higher fidelity concept representations that enable more balanced inference-time interventions, such as steering emotional tone while maintaining text coherence.
Key insights
- Concepts within LLMs are more accurately described as multidimensional subspaces rather than single, brittle vectors, reflecting their inherent fuzziness and context-dependency.
- Modeling concepts with Gaussian distributions provides a statistically more robust and stable representation, enhancing the reliability of interpretability findings.
- The GCS framework can reveal plausible hierarchical relationships between concepts, indicating that LLMs organize knowledge in semantically coherent ways that mirror human understanding.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.