Reproduce “HIPTrack: Visual Tracking with Historical Prompts” on OpenResearch

About this paper

Wenrui Cai, Qingjie Liu, Yunhong Wang

HIPTrack introduces a historical prompt network to comprehensively leverage past target features and precise positional information within Siamese visual trackers. This method attains state-of-the-art performance on LaSOT (72.7% AUC), GOT-10k (77.4% AO), and TrackingNet (84.5% AUC), while maintaining high inference speed (45.3 FPS) and significantly reducing trainable parameters (34.1M) compared to previous approaches.

Key insights

• Providing precise and updated historical information, encompassing both visual features and accurate positional data, yields substantial performance improvements in Siamese trackers even when the core feature extraction backbone is frozen.
• Prompt learning, a concept adapted from NLP, can effectively enhance visual tracking by guiding the core similarity matching process with rich historical context, enabling adaptive feature enhancement.
• The combination of a predicted bounding box and a Candidate Elimination mask produces a highly accurate "refined foreground mask," which is crucial for precise encoding of historical target position information.
• An L2-distance based attention mechanism within the historical prompt decoder enables adaptive aggregation of relevant historical information, allowing the system to focus on specific past frames when needed.

Summary by alphaXiv.