Reproduce “Disentangling Geometry, Performance, and Training in Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Atharva Kulkarni, Jacob Mitchell Springer, Arjun Subramonian, Swabha Swayamdipta

Researchers from the University of Southern California, Carnegie Mellon University, and UCLA systematically investigated how language model performance relates to the geometry of its unembedding matrix, finding that effective rank primarily reflects training dynamics rather than consistently predicting performance across diverse tasks and configurations. The study also showed that the "saturation problem" observed in some small models is not a universal phenomenon tied solely to low effective rank.

Key insights

  • The effective rank of the unembedding matrix is strongly influenced by training hyperparameters and primarily reflects optimization dynamics, rather than serving as a consistent, direct predictor of task-relevant performance.
  • The "saturation problem," where small models experience late-stage performance degradation and effective rank collapse, is not a universal phenomenon and is not solely caused by low effective rank, suggesting architectural factors can play a more significant role.
  • Other geometric metrics and the geometry of learned representations offer limited additional predictive insight into performance compared to the effective rank of the unembedding matrix.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.