Reproduce “Instance Brownian Bridge as Texts for Open-vocabulary Video Instance Segmentation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zesen Cheng, Kehan Li, Hao Li, Peng Jin, Chang Liu, Xiawu Zheng, Rongrong Ji, Jie Chen

Instance Brownian Bridge as Texts for Open-vocabulary Video Instance Segmentation introduces BriVIS, a framework that models video instance movements as Brownian Bridges to integrate temporal dynamics into open-vocabulary video instance segmentation. This method achieved a 49.49% mAP improvement over prior state-of-the-art on the BURST dataset, demonstrating enhanced ability to recognize novel object categories in videos.

Key insights

  • Modeling instance trajectories as Brownian Bridges provides a robust framework for integrating temporal dynamics directly into instance representations, enhancing open-vocabulary video understanding.
  • Aligning an instance's entire trajectory (bridge-level) rather than individual frames significantly improves semantic precision and contextual understanding in videos.
  • The proposed contrastive objectives mathematically enforce Brownian bridge dynamics, enabling the vision-language model to learn discriminative, temporally-aware instance representations effectively aligned with textual descriptions.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.