Reproduce “D-Fusion: Direct Preference Optimization for Aligning Diffusion Models with Visually Consistent Samples” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zijing Hu, Fengda Zhang, Kun Kuang
A method called D-Fusion from Zhejiang University and Nanyang Technological University generates visually consistent, reinforcement learning-trainable preference data for diffusion models. It enhances prompt-image alignment, leading to improved human preference scores and higher CLIP scores across various image generation tasks.
Key insights
- Visual consistency within preferred and dispreferred image pairs is critical for effective DPO training in diffusion models, analogous to the role of fine-grained data in language models.
- A self-attention based fusion process, guided by cross-attention masks, can effectively transfer semantic alignment characteristics while maintaining visual coherence across denoising steps.
- Preserving the full denoising trajectory of generated samples is essential for enabling reinforcement learning on diffusion models, as alternative methods like DDIM inversion prove insufficient.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.