Reproduce “Scene Consistency Representation Learning for Video Scene Segmentation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Haoqian Wu, Keyu Chen, Yanan Luo, Ruizhi Qiao, Bo Ren, Haozhe Liu, Weicheng Xie, Linlin Shen
Scene Consistency Representation Learning (SCRL) learns robust shot representations for video scene segmentation by identifying semantically consistent, potentially distant shots in unlabeled long-term videos. The method achieves state-of-the-art performance on the MovieNet-318 dataset.
Key insights
- Semantic consistency, even across temporally distant shots, is a more effective signal for learning robust scene representations than local temporal proximity.
- Combining cluster-based consistency with self-augmentation (Soft-SC) for positive sample generation yields superior representations compared to single-source methods.
- A boundary-free Bi-LSTM model is more suitable for evaluating scene-level representations than traditional MLP-based sliding window approaches, as it captures long-term dependencies with less inductive bias.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.