Reproduce “Expressivity of Spiking Neural Networks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Manjot Singh, Adalbert Fono, Gitta Kutyniok
A theoretical analysis of Linear Spiking Neural Networks (LSNNs) demonstrates their capacity to generate piecewise linear mappings, including discontinuous functions, and to emulate multi-layer ReLU Artificial Neural Networks with linear overhead in architectural complexity. The work also reveals that shallow LSNNs can achieve exponentially more linear regions than shallow ReLU ANNs, indicating distinct computational structures.
Key insights
- LSNNs intrinsically generate piecewise linear mappings and uniquely possess the capability to realize discontinuous functions, a property not shared by ReLU ANNs.
- LSNNs can emulate any multi-layer ReLU ANN, demonstrating equivalent expressive power for continuous piecewise linear functions with a linear increase in layers and neurons.
- Shallow LSNNs with positive weights can partition input space into an exponential number of linear regions, offering a distinct advantage over shallow ReLU ANNs which are limited to two.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.