Reproduce “The Role of Temporal Hierarchy in Spiking Neural Networks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Filippo Moro, Pau Vilimelis Aceituno, Laura Kriener, Melika Payvand
Researchers at the Institute of Neuroinformatics, UZH and ETH Zurich, demonstrated that explicitly imposing a fast-to-slow temporal hierarchy in Spiking Neural Networks (SNNs) layers, inspired by biological brains, acts as a beneficial inductive bias. This approach consistently improved performance by up to 4.1% on the SHD dataset while achieving comparable accuracy with a 6x reduction in parameters, and this hierarchy naturally emerged during training.
Key insights
- Imposing a fast-to-slow temporal hierarchy serves as a beneficial inductive bias, improving SNN performance on tasks requiring temporal processing.
- Temporal hierarchies naturally emerge in SNNs when temporal parameters are optimized through gradient descent, even starting from homogeneous initializations.
- The advantages of temporal hierarchy are task-dependent, providing benefits only for tasks where temporal dynamics carry meaningful information.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.