Reproduce “SENTRY: SAM2-Enhanced Neighbor-Aware and Temporally Reasoned Memory for Visual Tracking” on OpenResearch

About this paper

Mohamad Alansari, Yonathan Michael, Hasan AlMarzouqi, Muzammal Naseer, Naoufel Werghi, Sajid Javed

SENTRY introduces a training-free module for SAM2-based visual object tracking, which validates mask updates for short-horizon temporal consistency before committing them to memory. This method significantly reduces erroneous memory writes, leading to new zero-shot state-of-the-art performance on benchmarks such as LaSOT, GOT-10k, VOT20, VOT22, and DiDi.

Key insights

• Relying solely on high-confidence masks for memory updates in visual trackers leads to accumulation of errors and tracking drift, especially in challenging scenarios.
• Explicitly validating candidate masks for short-horizon temporal consistency at the memory-admission stage significantly improves tracking robustness by preventing erroneous information from entering the memory.
• Incorporating "neighbor-aware" checks by comparing candidate trajectories against both the target and potential distractors helps disambiguate similar objects and prevent identity swaps, proving generalizable across various memory-augmented systems.

Summary by alphaXiv.