Reproduce “UNINEXT-Cutie: The 1st Solution for LSVOS Challenge RVOS Track” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Hao Fang, Feiyu Pan, Xiankai Lu, Wei Zhang, Runmin Cong

The UNINEXT-Cutie pipeline achieved 1st place in the LSVOS Challenge RVOS Track, scoring 62.57% J&F on the challenging MeViS dataset by effectively combining a state-of-the-art referring video object segmentation model with a video object segmentation model and semi-supervised learning to track objects based on motion descriptions. This work from Shandong University's MVP-TIME team sets a new benchmark for understanding dynamic object behaviors within video based on linguistic prompts.

Key insights

  • Combining state-of-the-art RVOS (UNINEXT) and VOS (Cutie) models synergistically improves performance, especially for temporal consistency in motion-centric video segmentation.
  • Post-processing with a VOS model, including multi-mask inference and fusion via voting, significantly enhances mask quality and temporal coherence from initial RVOS outputs.
  • Semi-supervised learning, leveraging high-quality pseudo-labels generated by the refined pipeline, provides further substantial performance gains for model fine-tuning.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.