Reproduce “Hebbian Deep Learning Without Feedback” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Adrien Journé, Hector Garcia Rodriguez, Qinghai Guo, Timoleon Moraitis

The paper introduces Multilayer SoftHebb, an unsupervised, purely Hebbian learning algorithm that scales to deep networks of up to five hidden layers. This system achieves competitive classification accuracies (e.g., 80.3% on CIFAR-10) while addressing backpropagation's limitations such as the weight transport problem and non-local plasticity, offering superior biological plausibility and computational efficiency.

Key insights

  • A new learning regime with an optimal temperature (around τ=1) in SoftHebb, combined with soft anti-Hebbian plasticity, allows for the coexistence of diverse features (simple and complex), leading to maximal accuracy.
  • The novel weight-norm-dependent adaptive learning rate significantly stabilizes and speeds up learning, enabling the entire deep network to converge and achieve high performance after just a single epoch of unsupervised training.
  • Deep, purely Hebbian networks can effectively learn hierarchical representations, evidenced by progressively more complex and abstract receptive fields in deeper layers and improved class separation in embedding spaces.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.