Reproduce “SDTrack: A Baseline for Event-based Tracking via Spiking Neural Networks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yimeng Shan, Zhenbang Ren, Haodi Wu, Wenjie Wei, Rui-Jie Zhu, Shuai Wang, Dehao Zhang, Yichen Xiao and 6 more
The paper introduces SDTrack, an energy-efficient, fully spike-driven Transformer-based pipeline for event-based object tracking, incorporating a novel event aggregation method called Global Trajectory Prompt (GTP). SDTrack achieves state-of-the-art tracking accuracy on multiple event-based benchmarks while significantly reducing energy consumption and parameter count compared to existing methods.
Key insights
- Fully spike-driven Transformer architectures can achieve competitive and state-of-the-art performance in event-based tracking with significantly lower energy consumption than ANN-based or hybrid methods.
- Robust event aggregation, like GTP, which captures detailed spatiotemporal cues and provides a three-channel output compatible with pre-trained vision models, is crucial for maximizing tracker performance.
- Spiking Neural Networks can effectively learn positional information through specialized input formats (IPL) without relying on explicit positional encodings, simplifying architecture and improving robustness.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.