Reproduce “Kernelized information bottleneck leads to biologically plausible 3-factor Hebbian learning in deep networks” on OpenResearch

About this paper

Roman Pogodin, Peter E. Latham

The authors propose layer-wise learning that replaces backpropagated gradients with pair-based updates driven by local activity and label similarity. A Gaussian-kernel plausible HSIC rule implements this three-factor mechanism without weight transport or a backward pass. On CIFAR-10, it reached 90.4% versus 90.9% for matched backpropagation in a 2×-wide network, but the work remains a simulation with biological limitations.

Key insights

• The pHSIC approximation maintains sufficient fidelity to the original information bottleneck objective, allowing for biologically plausible online learning without significant performance degradation.
• The mathematical derivation of the learning rule under specific kernel choices and network architectures naturally incorporates divisive normalization, suggesting its functional importance in biologically realistic learning.
• The proposed 3-factor Hebbian rule, driven by fluctuations or differences in activity rather than absolute levels, offers a mechanism for synaptic plasticity that aligns with neurophysiological models involving global and local modulatory signals.

Summary by alphaXiv.