Reproduce “SOTFormer: A Minimal Transformer for Unified Object Tracking and Trajectory Prediction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhongping Dong, Pengyang Yu, Shuangjian Li, Liming Chen, Mohand Tahar Kechadi
SOTFormer unifies single-object tracking and short-horizon trajectory prediction into a constant-memory Transformer architecture, achieving 76.3% AUC on Mini-LaSOT while operating at 50 FPS with 4.3 GB VRAM. This approach eliminates memory growth with sequence length and improves tracking stability compared to previous Transformer-based methods.
Key insights
- Explicitly detaching the memory tensor in a Transformer allows for constant GPU memory usage regardless of video sequence length, enabling efficient long-term tracking.
- Deterministic initialization via ground-truth priming and an anchor loss significantly mitigates cold-start ambiguity and improves early localization stability in DETR-based trackers.
- Integrating motion forecasting gradients into spatial reasoning within a unified loss fosters greater temporal coherence and robust understanding of object dynamics.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.