Reproduce “Hessian Aware Quantization of Spiking Neural Networks” on OpenResearch

About this paper

Hin Wai Lui, Emre Neftci

Researchers at the University of California, Irvine, introduced a Hessian-aware quantization strategy combined with a simplified neuron model for Spiking Neural Networks (SNNs) to enhance their efficiency on neuromorphic hardware. This approach enabled a 58% reduction in network size while maintaining 97.8% accuracy on the N-MNIST dataset by intelligently allocating bit-precisions based on layer sensitivity.

Key insights

• The Hessian trace effectively quantifies a layer's sensitivity to weight perturbations, providing a reliable metric for determining optimal bit-precision allocation in mixed-precision quantization.
• Simplifying neuron models can drastically reduce memory and computational requirements for SNN inference without compromising the ability to train them with gradient-based methods.
• Layers contributing more to overall network accuracy typically exhibit higher Hessian traces, indicating their greater sensitivity to numerical precision and justifying higher bit-precision assignments.

Summary by alphaXiv.