Reproduce “Swapping Autoencoder for Deep Image Manipulation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Taesung Park, Jun-Yan Zhu, Oliver Wang, Jingwan Lu, Eli Shechtman, Alexei A. Efros, Richard Zhang
Researchers from UC Berkeley and Adobe Research developed the Swapping Autoencoder, a deep generative model that disentangles image structure and texture in an unsupervised manner for controllable manipulation of existing images. This approach achieves real-time image embedding and produces highly photorealistic hybrid images, demonstrating superior perceptual quality in human evaluations and enabling diverse editing functionalities.
Key insights
- Designing asymmetric latent codes (spatial for structure, global vector for texture) within an autoencoder framework, combined with cross-image swapping, inherently encourages disentanglement of these intuitive image components.
- The co-occurrence patch discriminator is crucial for ensuring texture consistency and photorealism during swapping, explicitly guiding the model to accurately transfer fine-grained texture while preserving the underlying structure.
- An efficient autoencoder design enables real-time image embedding (single feedforward pass), overcoming the computational burden of iterative optimization methods used for GAN inversion and making interactive editing practical.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.