Reproduce “IMPUS: Image Morphing with Perceptually-Uniform Sampling Using Diffusion Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zhaoyuan Yang, Zhengyang Yu, Zhiwei Xu, Jaskirat Singh, Jing Zhang, Dylan Campbell, Peter Tu, Richard Hartley

IMPUS (Image Morphing with Perceptually-Uniform Sampling Using Diffusion Models) introduces a diffusion-based framework for generating smooth, direct, and realistic morphing sequences between arbitrary image pairs. It leverages optimized text embeddings, spherical latent state interpolation, adaptive LoRA fine-tuning guided by a novel relative perceptual path diversity score, and perceptually-uniform sampling, outperforming prior diffusion-based morphing methods in smoothness and realism.

Key insights

  • Directly interpolating optimized text embeddings and deterministic Gaussian latent states within a diffusion model effectively bridges semantic and structural differences between arbitrary images.
  • Adaptive, low-rank fine-tuning guided by a perceptual path diversity metric allows for dynamic control over morph directness and realism without extensive full model training.
  • Perceptually-uniform sampling, rather than uniform alpha-space sampling, is critical for achieving visually smooth and consistent transitions in the generated morphing sequences.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.