Reproduce “From Data Statistics to Feature Geometry: How Correlations Shape Superposition” on OpenResearch

About this paper

Lucas Prieto, Edward Stevinson, Melih Barsbey, Tolga Birdal, Pedro A.M. Mediano

Researchers from Imperial College London developed a framework demonstrating how feature correlations in data lead to "linear superposition" in neural networks, explaining the emergence of semantic clusters and cyclical structures observed in large language models. The work shows that constructive interference from correlated features, rather than just interference filtering, shapes feature geometry, especially under capacity constraints.

Key insights

• Feature correlations in data can lead to constructive interference, where co-activating features reinforce each other, enabling efficient reconstruction with lower weight norms under tight latent bottlenecks or weight decay.
• The observed semantic clusters and cyclical structures in large language models can be mechanistically explained by models learning linear superposition that directly mirrors the inherent covariance of correlated input features.
• A conceptual distinction between "presence-coding" (discrete, correlation-driven) and "value-coding" (continuous, task-driven) features offers a principled approach to categorize different forms of feature geometry.

Summary by alphaXiv.