Reproduce “LucidFlux: Caption-Free Photo-Realistic Image Restoration via a Large-Scale Diffusion Transformer” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Song Fei, Tian Ye, Lujia Wang, Lei Zhu
LucidFlux restores degraded photos by adapting the 12-billion-parameter Flux.1 with low-quality and lightly restored image views, separate conditioner branches, timestep-and-layer modulation, and SigLIP semantic features instead of generated captions. On RealLQ250, it reaches 0.7406 CLIP-IQA+, versus 0.7034 for SeeSR. However, it does not lead every metric and cannot guarantee pixel-faithful recovery.
Key insights
- Large, frozen diffusion transformers can be effectively adapted for complex downstream tasks like image restoration using lightweight, harmoniously integrated conditioning modules without compromising the backbone's pre-trained capabilities.
- Caption-free semantic guidance, derived directly from image features, offers a superior and more efficient alternative to text-based methods by mitigating issues of latency, semantic drift, and degradation-related bias.
- A carefully curated, large-scale, and diverse dataset is crucial for unlocking the full potential and generalization ability of billion-parameter generative models in real-world applications.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.