Reproduce “Self-Supervised Video Similarity Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Giorgos Kordopatis-Zilos, Giorgos Tolias, Christos Tzelepis, Ioannis Kompatsiaris, Ioannis Patras, Symeon Papadopoulos
Researchers from Czech Technical University in Prague, ITI-CERTH, and Queen Mary University of London developed S2VS, a self-supervised learning framework for video similarity that achieves state-of-the-art performance on various retrieval and detection tasks without requiring labeled training data. The method matches or surpasses supervised baselines in mAP on datasets like VCDB and FIVR-200K, and shows over 15% improvement in detection 5AP on challenging subsets, demonstrating robust similarity calibration.
Key insights
- Self-supervised learning, particularly instance-discrimination with carefully designed augmentations and loss functions, can train a generalizable video similarity model that performs comparably or better than supervised methods without any labeled data.
- A diverse set of strong augmentations (global, frame, temporal like Temporal Shuffle-Dropout, and Video-in-Video) is crucial for learning robust representations capable of handling various real-world video transformations.
- The proposed Self-Similarity Hard-Negative loss ($L_{sshn}$) is critical for effectively calibrating similarity scores, leading to superior performance in video detection tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.