Reproduce “Diversity Has Always Been There in Your Visual Autoregressive Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Tong Wang, Guanyu Yang, Nian Liu, Kai Wang, Yaxing Wang, Abdelrahman M Shaker, Salman Khan, Fahad Shahbaz Khan and 1 more

This work introduces DiverseVAR, a training-free framework that effectively addresses diversity collapse in Visual Autoregressive (VAR) models during inference. It significantly enhances generative diversity, achieving higher Recall and Coverage and lower FID scores (e.g., FID for Infinity-2B drops from 28.48 to 22.96 on COCO 2014), while preserving image quality and text-image alignment.

Key insights

  • The fundamental structural elements of an image are primarily formed during the early scales of the VAR generation process.
  • Generative diversity is largely governed by a 'pivotal component' (dominant singular values) within the feature map, while 'auxiliary tokens' primarily contribute to semantic fidelity.
  • Targeted, training-free manipulation of these pivotal components at early stages of inference can unlock latent diversity without compromising image quality or semantic alignment.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.