Reproduce “Emergent Symbol-like Number Variables in Artificial Neural Networks” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Satchel Grant, Noah D. Goodman, James L. McClelland

This research explores how artificial neural networks develop numeric representations, demonstrating that recurrent networks trained on quantity matching tasks spontaneously form graded, symbol-like variables consistent with "Up-Down" algorithmic strategies. It also reveals that shallow Transformers rely on anti-Markovian re-computation for numeric tasks, rather than maintaining cumulative hidden states.

Key insights

  • Neural networks, without explicit numeric labels, can spontaneously form graded, symbol-like numeric variables for quantity tracking, exhibiting characteristics observed in human numeric cognition.
  • Network architecture (e.g., recurrent vs. transformer) and specific task parameters critically influence whether models adopt Markovian cumulative state strategies or anti-Markovian re-computation.
  • Causal interventions on distributed neural subspaces, particularly using Linear Alignment Functions, provide a robust method for interpreting neural mechanisms that is superior to individual neuron analysis.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.