Reproduce “DINO-CoDT: Multi-class Collaborative Detection and Tracking with Vision Foundation Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xunjie He, Christina Dao Wen Lee, Meiling Wang, Chengran Yuan, Zefan Huang, Yufeng Yue, Marcelo H. Ang Jr
The DINO-CoDT framework enables multi-class collaborative 3D detection and tracking by enhancing feature fusion from multiple agents and modalities, and leveraging vision foundation models for robust tracklet re-identification. It achieved average accuracy gains of 6.8% in [email protected] and 8.3% in [email protected] on the V2X-Real dataset for diverse road users, while reducing identity switches.
Key insights
- Enhanced multi-scale feature learning and inter-agent fusion, particularly with the GSAF module, significantly improves the detection of diverse objects, including small targets, in BEV representations.
- Vision Foundation Models like DINOv2 can extract robust visual semantics, effectively reducing identity switches in collaborative multi-object tracking by enabling appearance-based re-identification.
- Adaptive tracklet management and class-specific motion models are crucial for maintaining consistent identities and optimizing tracklet lifecycle across objects with varied motion characteristics.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.