Reproduce “SDTrack: A Baseline for Event-based Tracking via Spiking Neural Networks” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yimeng Shan, Zhenbang Ren, Haodi Wu, Wenjie Wei, Rui-Jie Zhu, Shuai Wang, Dehao Zhang, Yichen Xiao and 6 more

The paper introduces SDTrack, an energy-efficient, fully spike-driven Transformer-based pipeline for event-based object tracking, incorporating a novel event aggregation method called Global Trajectory Prompt (GTP). SDTrack achieves state-of-the-art tracking accuracy on multiple event-based benchmarks while significantly reducing energy consumption and parameter count compared to existing methods.

Key insights

  • Fully spike-driven Transformer architectures can achieve competitive and state-of-the-art performance in event-based tracking with significantly lower energy consumption than ANN-based or hybrid methods.
  • Robust event aggregation, like GTP, which captures detailed spatiotemporal cues and provides a three-channel output compatible with pre-trained vision models, is crucial for maximizing tracker performance.
  • Spiking Neural Networks can effectively learn positional information through specialized input formats (IPL) without relying on explicit positional encodings, simplifying architecture and improving robustness.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.