Reproduce “Quadratic integrate-and-fire neurons exhibit less fragmented loss landscapes and outperform leaky integrate-and-fire neurons in spike-based gradient descent” on OpenResearch

About this paper

Carlo Wenig, Raoul-Martin Memmesheimer, Christian Klos

Researchers from the University of Bonn and partners empirically demonstrated that Quadratic Integrate-and-Fire (QIF) neurons lead to less fragmented loss landscapes and significantly higher accuracy in Spiking Neural Networks (SNNs) trained with spike-based gradient descent compared to Leaky Integrate-and-Fire (LIF) neurons, achieving 90.1% accuracy versus 79.2% on the SHD dataset.

Key insights

• Neuron models with provably continuous spiking dynamics, such as QIF, translate their theoretical advantages into concrete practical benefits for SNN training performance and stability.
• QIF neurons lead to less fragmented and more continuous loss landscapes, facilitating more effective and predictable gradient-based optimization compared to the highly discontinuous landscapes of LIF neurons.
• Gradients derived from QIF networks are significantly more predictive of actual loss changes during training, making them a more reliable signal for gradient descent algorithms.

Summary by alphaXiv.