Reproduce “Spatial Gram Alignment for Ultra-High-Resolution Image Synthesis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jinjin Zhang, Xiefan Guo, Di Huang
Researchers from Beihang University developed Spatial Gram Alignment (SGA), a framework designed to fine-tune pre-trained Latent Diffusion Models for ultra-high-resolution image synthesis without degrading fine-grained detail. SGA aligns internal spatial self-similarities between generative features and vision foundation priors, achieving state-of-the-art performance on 2K and 4K benchmarks with notable improvements in metrics like gFID.
Key insights
- Direct patch-wise feature distillation, a common component of REPA, disrupts pre-trained LDM generative manifolds, causing significant degradation in fine-grained detail synthesis when scaling to ultra-high resolutions.
- Aligning internal spatial self-similarities (Gram matrices) enables generative models to leverage deep semantic priors for improved global structure while preserving the intrinsic high-frequency details of pre-trained LDMs.
- SGA's channel-orthogonal gauge invariance and a strictly larger zero-loss set than direct feature matching explain its non-invasive nature, allowing for greater flexibility in feature representation while still achieving alignment.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.