Reproduce “Network Plasticity as Bayesian Inference” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
David Kappel, Stefan Habenschuss, Robert Legenstein, Wolfgang Maass
The authors propose a stochastic plasticity mechanism that samples network configurations using experience-dependent drift, a prior over parameters, and ongoing Wiener-process noise. The batch dynamics have the posterior as their unique stationary distribution under stated conditions, while the online approximation lacks a convergence guarantee. In simulations, a bimodal prior preserved held-out performance during prolonged training on five MNIST digit-one examples.
Key insights
- Inherent biological stochasticity (e.g., spine motility, protein dynamics) is not mere noise but a functionally essential mechanism enabling neural networks to continuously sample from a posterior distribution of parameters, facilitating probabilistic inference.
- The derived SDE provides a normative model demonstrating how local, stochastic plasticity rules can enable a neural network to maintain a stationary distribution of its parameters that corresponds to a Bayesian posterior.
- Plasticity is reinterpreted as continuous movement of network parameters within a functional compromise manifold, rather than a deterministic convergence to single optimal values.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.