Reproduce “Face2Scene: Using Facial Degradation as an Oracle for Diffusion-Based Scene Restoration” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Amirhossein Kazerouni, Maitreya Suin, Tristan Aumentado-Armstrong, Sina Honari, Amanpreet Walia, Iqbal Mohomed, Konstantinos G. Derpanis, Babak Taati and 1 more

Face2Scene introduces a two-stage framework that utilizes a high-quality restored face as an oracle to estimate scene-wide image degradation, then conditions a one-step diffusion model for full-scene restoration. The approach consistently outperforms state-of-the-art methods on synthetic and real-world datasets across perceptual and fidelity metrics, processing 1024x1024 images in approximately 3.3 seconds.

Key insights

  • A high-quality restoration of a structured image patch like a human face can serve as a robust "perceptual oracle" for scene-wide degradation, as it provides a direct measure of corruption.
  • Learning a degradation representation by comparing a low-quality face with its high-quality, reference-based reconstruction effectively disentangles degradation from image content.
  • Providing multi-scale, explicit degradation tokens as conditioning significantly improves the ability of diffusion models to restore full scenes with higher fidelity and perceptual quality.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.