Reproduce “Constrained belief updates explain geometric structures in transformer representations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mateusz Piotrowski, Paul M. Riechers, Daniel Filan, Adam S. Shai
The paper explains how small transformers predict sequences from a three-state hidden Markov model by approximating Bayesian belief updates under attention’s parallel constraints. Attention adds distance-weighted, token-specific corrections to a stationary belief, while the MLP warps this constrained fractal toward the full belief geometry. In tested Mess3 models, two heads recover alternating influence when transition eigenvalues are negative; results do not generalize to large language models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.