Reproduce “Self-Supervised Learning of Event-Based Optical Flow with Spiking Neural Networks” on OpenResearch

About this paper

Jesse Hagenaars, Federico Paredes-Vallés, Guido de Croon

Researchers from the Micro Air Vehicle Laboratory at Delft University of Technology developed a self-supervised learning framework enabling deep Spiking Neural Networks (SNNs) to estimate event-based optical flow. The SNNs process raw, asynchronous event streams directly and achieve performance competitive with state-of-the-art Artificial Neural Networks (ANNs) on a dense regression task, facilitating energy-efficient, low-latency perception.

Key insights

• Deep SNNs are capable of solving complex, dense regression tasks like optical flow, demonstrating their potential beyond classification when trained with suitable methodologies.
• The choice of surrogate gradient width and careful initialization of the non-spiking prediction layer are crucial for stable convergence and preventing gradient vanishing in deep SNNs.
• Presynaptic adaptive neuron models (PLIF, XLIF) and learnable membrane leaks prove more effective for processing sparse, fast-changing event data than postsynaptic adaptation (ALIF), optimizing temporal integration within the network.

Summary by alphaXiv.