Reproduce “Incremental learning for audio classification with Hebbian Deep Neural Networks” on OpenResearch

About this paper

Riccardo Casciotti, Francesco De Santis, Alberto Antonietti, Annamaria Mesaros

Researchers from Tampere University and Politecnico di Milano developed a biologically inspired solution for continual learning in audio classification using Hebbian deep neural networks combined with a novel kernel plasticity mechanism. This approach reduced catastrophic forgetting, achieving 76.3% overall accuracy on the ESC-50 dataset, an improvement of 7.6 percentage points over a baseline without kernel plasticity.

Key insights

• Hebbian learning, augmented with a targeted kernel plasticity mechanism, can serve as an effective and biologically plausible alternative for mitigating catastrophic forgetting in continual learning.
• Selectively modulating learning rates of kernels based on their historical importance and current activity effectively balances the stability-plasticity trade-off, enabling robust incremental learning.
• An unsupervised Hebbian feature extractor, when combined with supervised task-specific classification heads, provides a viable architectural approach for continual learning without relying solely on backpropagation.
• The proposed neuro-modulation inspired kernel plasticity protects previously acquired knowledge while maintaining the capacity to learn new tasks efficiently.

Summary by alphaXiv.