Reproduce “Rectify Then Diffuse: Disentangling Concepts Before Denoising Trajectory Unfolds” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ning Zhu, An Chen, Mengfei Zhao, Juntao Xu, Jingze Liang, Boyuan Gu, Liang-Jian Deng
Rectify-then-Diffuse (RTD) introduces a training-free framework to resolve compositional failures in text-to-image diffusion models by disentangling conflicting concepts within the initial latent state before denoising. This method improves compositional fidelity, evidenced by state-of-the-art BLIP-VQA (0.7503) and ImageReward (1.2144) scores on the AE-Bench O-O subset, while incurring a minimal 6.3% inference overhead.
Key insights
- Compositional failures in text-to-image models stem from initial concept conflicts detectable in the high-noise latent space, prior to significant image formation.
- A single, early intervention on the initial latent state can effectively disentangle concepts, offering a more fundamental and efficient solution than continuous denoising-time modifications.
- The proposed isotropic gradient rectification allows for a stable and predictable correction of concept overlap without needing to explicitly define target layouts or iterative updates.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.