Reproduce “Emergence and Localisation of Semantic Role Circuits in LLMs” on OpenResearch

About this paper

Nura Aljaafari, Danilo S. Carvalho, André Freitas

Researchers from the University of Manchester introduced COMPASS, a causal-temporal methodology, to investigate the mechanistic interpretability of semantic role circuits in Large Language Models (LLMs). This framework revealed that LLMs develop compact, causally functional subgraphs for abstract semantic roles, which emerge gradually and exhibit partial transferability across model scales and architectures.

Key insights

• Semantic role information localizes into compact, sparse, and role-specific subgraphs within LLMs, indicating dedicated computational mechanisms for abstract semantic structures.
• The structural organization of these semantic circuits stabilizes gradually over training, but their functional indispensability can emerge earlier and exhibit more volatile trajectories, indicating a structure-function dissociation.
• LLMs tend to reuse similar high-importance components (nodes) for semantic roles across scales and architectures, but implement divergent connection patterns (edges), while still maintaining similar high-level information-flow geometry.

Summary by alphaXiv.