Reproduce “How To Train Your Deep Multi-Object Tracker” on OpenResearch

About this paper

Yihong Xu, Aljosa Osep, Yutong Ban, Radu Horaud, Laura Leal-Taixe, Xavier Alameda-Pineda

This paper introduces a differentiable framework for end-to-end training of deep multi-object trackers by making standard evaluation metrics (MOTA, MOTP) differentiable. The approach, termed DeepMOT, leverages a Deep Hungarian Net to approximate non-differentiable assignment and achieves new state-of-the-art performance on MOTChallenge benchmarks, improving MOTA and reducing ID switches.

Key insights

• It is feasible to create differentiable approximations for traditionally non-differentiable combinatorial optimization problems (like optimal bipartite matching) using deep neural networks, enabling gradient flow.
• Directly optimizing deep multi-object trackers for standard evaluation metrics (like MOTA and MOTP) leads to improved performance compared to training with proxy losses.
• The framework allows for the joint, end-to-end training of all relevant components of a deep tracker, including appearance embedding modules for re-identification, leading to more robust and identity-preserving tracking.

Summary by alphaXiv.