Reproduce “Detecting AutoEncoder is Enough to Catch LDM Generated Images” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dmitry Vesnin, Dmitry Levshun, Andrey Chechulin
The work from St. Petersburg Federal Research Center of the Russian Academy of Sciences introduces a method to identify Latent Diffusion Model (LDM) generated images by training a classifier solely on artifacts introduced when real images are reconstructed by an LDM's autoencoder. This approach achieves high true positive rates and exceptionally low false positive rates against a diverse set of LDM-generated images, outperforming compared state-of-the-art methods.
Key insights
- It is sufficient to train a detector on the characteristic artifacts introduced by the autoencoder component of Latent Diffusion Models (LDMs) rather than on fully generated synthetic images.
- Autoencoders, irrespective of their specific variant (e.g., VAE vs. MoVQ) or the subsequent diffusion process, introduce consistent, detectable "fingerprints" onto images.
- Training on autoencoder reconstruction differences significantly enhances generalization capability, allowing detectors to perform well on unseen LDM architectures and proprietary models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.