Reproduce “SyncDiffusion: Coherent Montage via Synchronized Joint Diffusions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuseung Lee, Kunho Kim, Hyunjin Kim, Minhyuk Sung
SyncDiffusion guides panorama generation toward coherence by aligning predicted denoised views during diffusion. Building on MultiDiffusion’s averaging of overlapping windows, it adjusts views toward an anchor using perceptual-similarity guidance. In text-guided panoramas generated with Stable Diffusion 2.0, participants preferred SyncDiffusion over MultiDiffusion for coherence, 66.35% to 33.65%; the method adds computation and depends on suitable prompts.
Key insights
- Directly applying perceptual similarity metrics to noisy intermediate latent images during diffusion provides ineffective guidance.
- Utilizing the diffusion model's prediction of the clean image (`x_0`) at each denoising step yields informative observations, allowing perceptual loss to generate meaningful gradients for coherence.
- Combining this perceptual guidance with the latent feature averaging of joint diffusion simultaneously addresses both global coherence and local seamlessness.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.