Reproduce “Unsupervised Learning by Competing Hidden Units” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dmitry Krotov, John Hopfield
Krotov and Hopfield developed a local, unsupervised, and biologically plausible learning rule for training early layer feature detectors in neural networks. This method achieved a 1.52% test error on MNIST, comparable to supervised backpropagation, and a 49.25% test error on CIFAR-10, outperforming prior biologically-inspired algorithms.
Key insights
- Useful early layer representations, competitive with those learned by supervised backpropagation, can be acquired through purely local, unsupervised, and biologically plausible mechanisms.
- The inclusion of anti-Hebbian components in the learning rule allows for the development of negative weights, contributing to more discriminative feature detectors that learn both what 'is' and what 'is not' present.
- The proposed learning dynamics are proven to converge due to the existence of a Lyapunov function, providing theoretical stability guarantees.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.