Reproduce “CLIF: Complementary Leaky Integrate-and-Fire Neuron for Spiking Neural Networks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yulong Huang, Xiaopeng Lin, Hongwei Ren, Haotian Fu, Yue Zhou, Zunchang Liu, Biao Pan, Bojun Cheng
Researchers from The Hong Kong University of Science and Technology (Guangzhou) and Beihang University introduced the Complementary Leaky Integrate-and-Fire (CLIF) neuron, a new SNN model that addresses vanishing temporal gradients during training without adding learnable parameters. CLIF-based SNNs achieved 96.69% accuracy on CIFAR-10, slightly surpassing ReLU-based ANNs, and demonstrated competitive performance on other static image and neuromorphic datasets while maintaining energy efficiency.
Key insights
- The rapid vanishing of temporal gradients in LIF neurons is due to both an exponential decay factor and the instantaneous cutoff of gradient paths when a neuron spikes.
- The CLIF neuron model inherently generates additional, non-decaying temporal gradient pathways during backpropagation, effectively compensating for the vanishing gradient issue in LIF models.
- CLIF improves the utilization of temporal information and enhances training convergence in SNNs without requiring any extra learnable parameters, thereby maintaining computational efficiency.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.