Reproduce “DiT-IC: Aligned Diffusion Transformer for Efficient Image Compression” on OpenResearch

About this paper

Junqi Shi, Ming Lu, Xingchen Li, Anle Ke, Ruiqi Zhang, Zhan Ma

Researchers at Nanjing University introduced DiT-IC, an Aligned Diffusion Transformer for image compression that enables one-step reconstruction from a 32x downscaled latent space, directly tackling the high computational and memory overhead of existing diffusion-based codecs. The system achieves competitive perceptual image quality with decoding times as low as 0.15s for 1024x1024 images and reduced memory usage, enabling high-resolution reconstruction on consumer GPUs.

Key insights

• Denoising in diffusion models can be adapted to spatially varying latent uncertainty by using a variance-guided pseudo-timestep, enabling efficient one-step reconstruction.
• The behavior of a multi-step diffusion process can be effectively distilled into a single forward pass for reconstruction by leveraging a self-supervised alignment with a frozen encoder's latent output.
• Leveraging the rich semantic and structural information within compressed latent representations provides more efficient and accurate conditioning for image reconstruction compared to traditional text-based prompts.

Summary by alphaXiv.