Reproduce “Inversion-DPO: Precise and Efficient Post-Training for Diffusion Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zejian Li, Yize Li, Chenye Meng, Zhongni Liu, Yang Ling, Shengyuan Zhang, Guang Yang, Changyuan Yang and 2 more
Inversion-DPO refines diffusion model post-training by integrating Direct Preference Optimization with DDIM inversion, achieving more precise trajectory estimation and a simplified loss function. This method improves image generation quality on text-to-image and compositional tasks, demonstrating over 2x faster convergence and superior performance across human preference and image consistency benchmarks.
Key insights
- DDIM inversion offers a deterministic and precise method to reconstruct diffusion trajectories, which significantly improves the accuracy of posterior approximation for DPO-based alignment.
- This precise trajectory estimation enables a fundamental simplification of the DPO loss function for diffusion models, transforming complex KL-divergence terms into more direct matching objectives, thus boosting optimization efficiency.
- Effective alignment for complex generative tasks like compositional image generation benefits from multi-objective reward scoring and dynamically derived preferences from structured datasets.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.