Reproduce “Not All Diffusion Model Activations Have Been Evaluated as Discriminative Features” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Benyuan Meng, Qianqian Xu, Zitai Wang, Xiaochun Cao, Qingming Huang
This work re-evaluates the selection of discriminative features from diffusion U-Nets, proposing a two-stage methodology that combines a property-driven qualitative filtering with a targeted quantitative comparison. The approach consistently improves performance across semantic correspondence and segmentation tasks, leveraging the full potential of advanced diffusion models like Stable Diffusion XL to achieve new state-of-the-art results, for instance, a 7.58 [email protected] img improvement over legacy methods on SDXL for semantic correspondence.
Key insights
- Diffusion U-Nets possess universal, architecturally-dependent properties (asymmetric diffusion noises, in-resolution granularity changes, locality without positional embeddings) that enable effective qualitative filtering of internal activations.
- Older diffusion feature selection methods are insufficient for modern diffusion models like SDXL, often leading to worse performance compared to older models due to overlooked activations and architectural shifts.
- Self-attention locality, while generally detrimental for features, can paradoxically suppress diffusion noises in certain up-stage activations, making otherwise unusable high-resolution features valuable.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.