Reproduce “SOAR: Self-Correction for Optimal Alignment and Refinement in Diffusion Models” on OpenResearch

About this paper

You Qin, Linqing Wang, Hao Fei, Roger Zimmermann, Liefeng Bo, Qinglin Lu, Chunyu Wang

Tencent Hunyuan researchers developed SOAR, a post-training method for diffusion models that enables self-correction of trajectory errors during the denoising process. This approach consistently improved compositional accuracy (e.g., GenEval from 0.70 to 0.78) and multiple model-based quality metrics like PickScore and Aesthetic scores without relying on reward models.

Key insights

• The analytical derivation of z0 as a consistent correction target is valid for bounded off-trajectory deviations when re-noising with the original noise (z1).
• Self-correction during high-noise stages, where semantic decisions are critical, leads to more substantial improvements in compositional accuracy and text rendering.
• Dense, on-policy supervision allows for stable and efficient learning, outperforming sparse-reward RL methods in specific reward optimization tasks without typical trade-offs.

Summary by alphaXiv.