Reproduce “DiffuseVAE: Efficient, Controllable and High-Fidelity Generation from Low-Dimensional Latents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kushagra Pandey, Avideep Mukherjee, Piyush Rai, Abhishek Kumar
DiffuseVAE introduces a generator-refiner framework that integrates Variational Autoencoders (VAEs) with Denoising Diffusion Probabilistic Models (DDPMs), enabling high-fidelity and controllable image generation from low-dimensional latent spaces. The model achieves competitive FID scores on standard benchmarks like CIFAR-10 and CelebA-HQ, significantly improving the speed-quality tradeoff compared to traditional diffusion models while preserving VAE's interpretable latent features.
Key insights
- The generator-refiner paradigm effectively combines the strengths of VAEs (interpretable latent space) and DDPMs (high-fidelity refinement) to address their respective limitations.
- Conditioning the diffusion process on low-dimensional VAE reconstructions significantly accelerates high-quality sample generation, allowing comparable results with substantially fewer inference steps.
- The VAE's latent space, when coupled with the diffusion refiner, provides fine-grained, semantically meaningful control for image manipulation via interpolation and vector arithmetic.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.