Reproduce “Exploiting Diffusion Prior for Task-driven Image Restoration” on OpenResearch

About this paper

Jaeha Kim, Junghun Oh, Kyoung Mu Lee

EDTR restores degraded images for downstream vision tasks by first applying a pixel-error-trained restorer, then adding mild noise to its output and briefly denoising with diffusion. It also trains with task-related feature losses. On PASCAL VOC2012 detection under synthetic Mixture-A degradation, four steps raise mAP from 22.3% to 33.4%, but evidence is limited to synthetic benchmarks and qualitative real-world examples.

Key insights

• Starting the diffusion process from a pre-restored, mildly noisy latent representation (partial diffusion) effectively incorporates valuable clues from low-quality images and improves task performance.
• Significantly reducing the number of denoising steps from conventional long-step processes (e.g., 50) to short-step (1-4 steps) is crucial for TDIR, as it prevents dilution of task-relevant information.
• Combining High-level Feature Loss and Feature Matching Loss is essential for guiding the diffusion model to restore details that are not only visually appealing but also critically beneficial for downstream high-level vision tasks.

Summary by alphaXiv.