Reproduce “A Tilted Seesaw: Revisiting Autoencoder Trade-off for Controllable Diffusion” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Pu Cao, Yiyang Ma, Feng Zhou, Xuedan Yin, Qing Song, Lu Yang
This study revisits the autoencoder trade-off within latent diffusion models, demonstrating that a prevailing bias towards generative quality metrics in autoencoder evaluation leads to models that introduce 'condition drift,' thereby hindering controllable image generation. The work establishes that instance-level reconstruction fidelity serves as a more reliable indicator of an autoencoder's ability to preserve control-relevant information for conditional image synthesis.
Key insights
- Generative FID (gFID) demonstrates consistently low correlation with control condition drift measures, indicating it is an unreliable metric for predicting an autoencoder's suitability for controllable generation.
- Instance-level reconstruction metrics, such as PSNR and SSIM, strongly correlate with and track control condition drift, serving as more effective safeguards and predictors of an autoencoder's ability to preserve control-relevant information.
- Theoretically, instance-level reconstruction error provides an upper bound on condition drift under reasonable assumptions, explaining its strong empirical relationship with preserved control.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.