Reproduce “Representational Curvature Modulates Behavioral Uncertainty in Large Language Models” on OpenResearch

About this paper

Jack King, Evelina Fedorenko, Eghbal A. Hosseini

Researchers from MIT demonstrate that representational curvature in large language models directly modulates behavioral uncertainty, showing that higher curvature correlates with increased next-token entropy. Their work reveals that targeted geometric perturbations and auxiliary curvature regularization can systematically alter model uncertainty without degrading overall predictive performance.

Key insights

• Representational curvature acts as a functional, task-aligned feature that modulates behavioral uncertainty in LLMs, directly linking an internal geometric property to an observable output characteristic.
• The coupling between representational straightening and predictive uncertainty naturally emerges and strengthens in intermediate layers over the course of LLM training.
• Geometric interventions, particularly those aligned with the model's representational trajectory, can reliably and selectively control model uncertainty, suggesting a mechanistic handle on LLM behavior.

Summary by alphaXiv.