Reproduce “Learning without feedback: Fixed random learning signals allow for feedforward training of deep neural networks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Charlotte Frenkel, Martin Lefebvre, David Bol
The Direct Random Target Projection (DRTP) algorithm enables purely feedforward training of deep neural networks by using fixed random learning signals from target labels, addressing backpropagation's weight transport and update locking problems to facilitate energy-efficient, on-chip learning for edge devices.
Key insights
- Simplified error signals, specifically the one-hot target labels, provide sufficient directional information for effective multi-layer network learning, challenging the necessity of precise error gradients.
- Decoupling network layers through direct target projection successfully solves both the weight transport and update locking problems simultaneously, a limitation of prior feedback alignment methods.
- The inherent low computational and memory overheads of DRTP make it highly suitable for adaptive online learning on resource-constrained edge computing devices.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.