Reproduce “Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades” on OpenResearch

About this paper

Garrick Orchard, Ajinkya Jayawant, Gregory Cohen, Nitish Thakor

Researchers from the Singapore Institute for Neurotechnology and collaborating institutions developed a methodology utilizing controlled saccadic movements of a neuromorphic sensor to convert static image datasets into biologically realistic event-based data. This effort yielded N-MNIST and N-Caltech101, which are the largest publicly available annotated neuromorphic datasets to date, effectively eliminating display artifacts and establishing new benchmarks for event-driven vision algorithms.

Key insights

• Controlled physical saccadic motion of a neuromorphic sensor is a highly effective and biologically plausible method for generating high-fidelity event-based data from static images.
• The proposed conversion technique successfully eliminates monitor refresh rate artifacts, as validated by Fourier analysis showing the absence of extraneous frequency peaks (e.g., 75Hz).
• Current spike-based recognition algorithms achieve significantly lower accuracies on these new datasets compared to traditional computer vision approaches, indicating substantial room for algorithmic improvement and validating the datasets' challenging nature.

Summary by alphaXiv.