Reproduce “Dual Learning with Dynamic Knowledge Distillation and Soft Alignment for Partially Relevant Video Retrieval” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jianfeng Dong, Lei Huang, Daizong Liu, Xianke Chen, Xun Yang, Changting Lin, Xun Wang, Meng Wang

Researchers introduce DL-DKD++, a framework for partially relevant video retrieval that employs dual learning with dynamic knowledge distillation from large vision-language models and dynamic soft alignment for nuanced relevance scoring. The approach establishes new state-of-the-art performance on datasets like TVR (SumR 184.8), ActivityNet-Captions (SumR 149.9), and Charades-STA.

Key insights

  • Leveraging powerful VL-PTMs for PRVR is most effective when their knowledge is dynamically distilled and adapted to mitigate domain gaps and temporal modeling challenges.
  • Modeling the varying degrees of partial relevance with dynamic soft targets significantly improves retrieval accuracy compared to rigid binary supervision.
  • A dual-branch student architecture, combining general knowledge inheritance with task-specific exploration, yields superior and more robust performance for untrimmed video retrieval.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.