Reproduce “Multi-Level Firing with Spiking DS-ResNet: Enabling Better and Deeper Directly-Trained Spiking Neural Networks” on OpenResearch

About this paper

Lang Feng, Qianhui Liu, Huajin Tang, De Ma, Gang Pan

Researchers from Zhejiang University and Zhejiang Lab introduce Multi-Level Firing (MLF) and a spiking Dormant-Suppressed Residual Network (DS-ResNet) to address gradient vanishing and network degradation in directly-trained Spiking Neural Networks (SNNs). The approach enables the training of deeper SNNs, achieving 94.25% accuracy on CIFAR10 and 97.29% on DVS-Gesture with reduced parameters and timesteps.

Key insights

• MLF effectively alleviates gradient vanishing by broadening the "gradient-available" region, ensuring more neurons contribute to the learning process and enabling faster convergence.
• The "activation before addition" design in spiking DS-ResNet significantly reduces the probability of a shortcut connection leading to dormant units and improves the network's ability to perform identity mapping with discrete spikes.
• The combined MLF and spiking DS-ResNet approach enables stable training of SNNs to much greater depths (e.g., 68 layers) without experiencing performance degradation, a limitation commonly found in prior directly-trained SNNs.

Summary by alphaXiv.