Reproduce “Revisiting Spectral Representations in Generative Diffusion Models” on OpenResearch

About this paper

Yuehao Wang, Peihao Wang, Hanwen Jiang, Ziyi Yang, Qixing Huang, Zhangyang Wang

The authors introduce a self-supervised spectral regularizer for diffusion models, aligning denoiser features across two independently noised views of each example. On ShapeNet Chair after 5K iterations, it lowers CD-based 1-NNA from 0.850 to 0.583 and raises COV from 0.295 to 0.488. It improves image FID over vanilla baselines, but REPA performs better on latent ImageNet-256.

Key insights

• Noise level determines the appropriate representation geometry: low-noise features benefit most from spectral organization, while high-noise representations become less discriminative as samples approach Gaussian noise.
• The proposed kernel links diffusion perturbations to diffusion maps and spectral clustering, providing a view of intermediate features as eigenfunctions of a noise-dependent data operator.
• Encoder-free alignment can improve generation in domains lacking foundation models, but it requires extra perturbed-view computations and does not consistently replace a strong external teacher.

Summary by alphaXiv.