Reproduce “Accelerated Predictive Coding Networks via Direct Kolen-Pollack Feedback Alignment” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Davide Casnici, Martin Lefebvre, Justin Dauwels, Charlotte Frenkel
Researchers at Delft University of Technology developed Direct Kolen–Pollack Predictive Coding (DKP-PC), an algorithm that integrates DKP feedback alignment with Predictive Coding to mitigate error delay and decay in deep neural networks. DKP-PC demonstrates comparable or superior classification accuracy to other local learning methods, achieving the highest Top-1 accuracy among them on Tiny ImageNet, while significantly reducing training time by up to 81% for convolutional networks compared to incremental PC.
Key insights
- DKP feedback matrices, under linear assumptions, converge to approximate a recursive Moore-Penrose pseudoinverse chain of forward weights, providing a theoretical basis for its improved alignment with Backpropagation.
- The single-step neural activity optimization in DKP-PC's inference phase acts as a regularizer for DKP updates, leading to faster, higher, and more stable gradient alignment with Backpropagation than standard DKP.
- DKP-PC achieves a theoretical time complexity of O(1) for error propagation, independent of network depth, by simultaneously addressing error delay and decay.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.