Reproduce “Tracking Representation Dynamics in Large Language Models with Persistent Homology” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Naman Malhotra, Jay Ambadkar, Abhinav Gupta, Kushal Kasivel, Abbas Schwarz, Kamillo Ferry, Anthea Monod

The authors track how language-model representations reorganize during LoRA fine-tuning by applying persistent homology to prompt activations across checkpoints. They measure topological velocity with the H₁ barcode Wasserstein distance and compare it with behavioral and geometric measures. Movement concentrates in the first third in 11 of 12 model–objective comparisons, but timing varies and magnitudes are model-normalized.

Key insights

  • LLMs undergo a universal, early, and transient phase of significant topological reorganization in their latent spaces during alignment fine-tuning, with the timing influenced by model scale.
  • Different alignment objectives (helpful, harmless, mixed) induce distinguishable patterns of topological evolution, primarily differing in the magnitude rather than the fundamental direction of change.
  • Internal representation dynamics, as revealed by persistent homology, can be decoupled from coarse behavioral metrics, indicating underlying 'hidden progress' or transformation within the latent space.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.