Reproduce “A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operations” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Bilal Chughtai, Lawrence Chan, Neel Nanda

Neural networks consistently learn a novel, mathematically derived Group Composition via Representations (GCR) algorithm for finite group operations. This work finds weak universality in the type of algorithm learned but refutes strong universality, as specific features and their learning order vary across different network initializations and architectures.

Key insights

  • The previously observed Fourier transform-based algorithm for modular addition is a specific instance of the more general GCR algorithm, which networks universally adopt for group composition.
  • Neural networks consistently implement the GCR algorithm through linear-algebraic operations: embeddings as lookup tables for representation matrices, hidden layers computing matrix products, and unembedding performing character calculations.
  • While networks learn the same *type* of algorithm (GCR), supporting weak universality, the *specific* irreducible representations chosen, their number, and their learning order vary arbitrarily across random seeds and architectures, refuting strong universality.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.