Reproduce “Scaling Forward Gradient With Local Losses” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mengye Ren, Simon Kornblith, Renjie Liao, Geoffrey Hinton
LG-FG-A uses neuron activity perturbations and local losses to reduce noisy forward-gradient learning. It divides signals among network blocks, image patches, and channel groups, while retaining backpropagation for the classifier. On supervised ImageNet, LocalMixer L/32/64 has 58.37% test error, versus 85.45% for LG-FA and 36.82% for backprop: better than the feedback-alignment baseline, but behind backprop.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.