Reproduce “End-to-End Self-Supervised RGB-T Tracking without Modality Misleading” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Shenglan Li, Rui Yao, Kunyang Sun, Hong Jia, Yong Zhou, Javen Qinfeng Shi, Xinyu Zhang

ESMTrack trains RGB-T trackers from the annotated first-frame box, forward tracking on unlabeled search frames, and backward tracking that filters samples for complementary triplet losses, without offline pseudo-label generation or dense search-frame box supervision. Its adaptive modality suppression uses APCE during training. On LasHeR, it reaches 47.3% PR and 36.0% SR; on RGBT234, PR is 70.6%, below GDSTrack’s 70.9%.

Key insights

  • Response-map peak quality (APCE) can guide modality balancing more directly than feature activation magnitude, which may be high for distractors as well as reliable evidence.
  • Forward–backward consistency provides a way to screen predicted locations for self-supervised training without treating generated boxes as direct regression targets.
  • Modality suppression is a training-time constraint, so the method aims to encourage complementary features without adding inference-time branches.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.