Reproduce “MAGREF: Masked Guidance for Any-Reference Video Generation with Subject Disentanglement” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yufan Deng, Yuanyang Yin, Xun Guo, Yizhi Wang, Jacob Zhiyuan Fang, Shenghai Yuan, Yiding Yang, Angtian Wang and 3 more

MAGREF generates videos from prompts and reference images, preserving distinct subjects. It marks regions on a shared canvas, binds each to matching prompt words, and supplies these features to the model. On a 60-case multi-subject benchmark, it scores 0.627 overall versus VACE’s 0.590, but does not lead every metric; this result is limited to that benchmark.

Key insights

  • Integrating multi-subject information into an Image-to-Video pipeline is effective through region-aware masking and pixel-wise channel concatenation, preserving fine-grained features without architectural modifications.
  • Explicitly binding each subject to its corresponding textual condition via targeted latent injection significantly reduces cross-subject interference and improves multi-subject video generation quality.
  • A systematic data curation strategy, including object/face processing and cross-pair data construction, is crucial for building robust generative models that handle complex multi-subject scenarios and generalize well.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.