Reproduce “Contrastive Hebbian Learning with Random Feedback Weights” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Georgios Detorakis, Travis Bartley, Emre Neftci
Random Contrastive Hebbian Learning (rCHL) is introduced as a biologically more plausible alternative to Contrastive Hebbian Learning (CHL) by replacing symmetric feedback weights with fixed random matrices. This method achieved classification accuracy comparable to backpropagation on MNIST (0.975) and solved XOR, while offering advantages for neuromorphic hardware by using continuous non-linear dynamics and local Hebbian updates.
Key insights
- Fixed random feedback matrices can effectively propagate error signals and enable learning in deep networks, demonstrating that synaptic symmetry is not strictly necessary for CHL-like learning dynamics.
- The spectral properties of the random feedback matrices, as analyzed by pseudospectra, directly correlate with learning dynamics; wider distributions or larger variances of matrix elements lead to faster and more stable convergence.
- Optimal feedback gain and learning rate parameters are crucial for rCHL, with a scaling factor in the weight update rule indicating that deeper networks may require stronger feedback signals to converge effectively.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.