Reproduce “NoiseTilt: Noise-Tilted Reverse Kernels for Diffusion Reward Alignment” on OpenResearch

About this paper

Jisung Hwang, Yunhong Min, Jaihoon Kim, I-Chao Shen, Minhyuk Sung

The Noise-Tilted Reverse Kernel (NTRK) steers pretrained diffusion models by whitening reward gradients and injecting them into each step’s noise while keeping the reverse-step mean and nominal noise scale fixed. On 45 animal prompts with FLUX, it scores 7.4510 in Aesthetic Score at 25 NFEs, above the best 500-NFE baseline; this compares evaluation budgets, not elapsed time or total cost.

Key insights

• Effective reward guidance can be achieved by perturbing the noise term of the diffusion reverse kernel rather than its mean, thus preserving the model's inherent statistical properties.
• Raw reward gradients are structured and must be 'whitened' to statistically resemble typical Gaussian noise before injection to prevent out-of-distribution artifacts and maintain sample quality.
• The proposed whitening operator, combining order-statistic projections and orthogonal transform domain projections, is critical for safely and effectively integrating gradient information into the noise channel.

Summary by alphaXiv.