Reproduce “An effective theory of collective deep learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Lluís Arola-Fernández, Lucas Lacasa

An effective theory for collective deep learning establishes that interacting neural networks exhibit depth-dependent disorder-order-disorder phase transitions, enabling individual units to achieve generalization across all data classes despite being trained only on private, isolated data. This approach leverages a coarse-grained model, mapping coupled network dynamics to a deformed Ginzburg-Landau model, and is validated through experiments on image classification.

Key insights

  • Collective learning in coupled neural networks can be understood as physical phase transitions, with depth ($D > 0$) leading to disorder-order-disorder transitions in the mean magnetization order parameter.
  • Neural network depth plays a critical role, delaying the onset of the collective learning phase and influencing the non-monotonicity of collective performance metrics.
  • Individual networks, despite being trained solely on private, single-class data, gain the ability to generalize fully to all unseen data classes once the system enters the collective learning phase.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.