Reproduce “Scalable Training of Continuous-Time Spiking Neural Networks with Differentiable Spike-Time Discretization” on OpenResearch

About this paper

Yusuke Sakemi, Tomoya Takeuchi, Takeo Hosomi, Kazuyuki Aihara

The authors introduce DSTD, mapping input spikes onto a fixed grid to approximate continuous-time dynamics while retaining continuous firing times. In time-to-first-spike networks, it reduces candidate-related activation memory from O(N_out N_in) to O(N_out M), while connection work remains. Experiments show 60–150-fold memory efficiency in a dense layer and 92.33% test accuracy for a 20-layer Fashion-MNIST model; firing-time errors can occur between grid points.

Key insights

• Approximating continuous spike dynamics with a fixed, differentiable set of discrete time points significantly reduces computational memory and runtime without substantial accuracy loss.
• Temporal regularization effectively mitigates the dead neuron problem and organizes spike propagation into sequential, layer-wise windows in deep SNNs.
• The organized temporal windows enable pipeline-like data processing, allowing deep SNNs to achieve higher throughput by processing new inputs before previous ones fully propagate.
• A trade-off exists between classification accuracy and processing latency (temporal window width) that can be optimized.

Summary by alphaXiv.