Reproduce “Transformers Are Born Biased: Structural Inductive Biases at Random Initialization and Their Practical Consequences” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Siquan Li, Yao Tong, Haonan Wang, Tianyang Hu

Researchers found that randomly initialized transformer models exhibit strong, seed-specific structural biases characterized by extreme token preferences, challenging the 'blank slate' assumption. This inherent architecture-driven bias, rooted in MLP asymmetric activations and amplified by self-attention, persists through training and enables novel LLM fingerprinting and principled mitigation of attention sinks.

Key insights

  • Randomly initialized transformers are not 'blank slates,' but instead possess strong, systematic, and seed-specific structural biases that manifest as extreme preferences for a tiny subset of output tokens.
  • The core mechanism driving these biases is an 'inter-sequence representation contraction' caused by asymmetric nonlinear activations in MLP sublayers, which is then significantly amplified by self-attention mechanisms.
  • Self-attention also induces 'intra-sequence contraction' and a systematic positional variance discrepancy, making the first token a structural outlier and causally linking this to the attention-sink phenomenon.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.