Reproduce “Ultra-High-Resolution Image Synthesis: Data, Method and Evaluation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jinjin Zhang, Qiuyu Huang, Junjie Liu, Xiefan Guo, Di Huang
Diffusion-4K introduces a comprehensive framework for direct ultra-high-resolution image generation at 4K, leveraging a new large-scale dataset, a memory-efficient VAE, and a detail-enhancing latent fine-tuning method. The framework achieves superior quantitative performance on new UHR metrics and receives higher human preference compared to existing methods like PixArt-Σ and Sana.
Key insights
- Scale Consistent Variational Auto-Encoder (SC-VAE) with scale consistency regularization allows efficient F=16 down-sampling while maintaining reconstruction quality, crucial for ultra-high-resolution VAEs.
- Wavelet-based Latent Fine-tuning (WLF) effectively guides diffusion models to generate rich high-frequency details by optimizing in the wavelet domain, overcoming limitations of traditional loss functions for UHR content.
- Traditional holistic metrics are insufficient for assessing ultra-high-resolution images; new local patch-level metrics like GLCM Score and Compression Ratio are necessary to capture texture richness and detail preservation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.