Reproduce “Robust ID-Specific Face Restoration via Alignment Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yushun Fang, Lu Liu, Xiang Gao, Qiang Hu, Ning Cao, Jianghe Cui, Gang Chen, Xiaoyun Zhang
The RIDFR framework introduces a novel alignment learning strategy with a pre-trained diffusion model to achieve robust identity-specific face restoration, consistently preserving identity and generating high visual quality even from highly degraded images and diverse reference photos. This approach effectively disentangles identity-relevant features from ID-irrelevant semantics, addressing a key challenge in generative face restoration.
Key insights
- Pixel-level channel-wise concatenation of the low-quality input in the Content Injection Module is more effective than latent-level methods, avoiding severe color shifts and information loss.
- The Identity Injection Module, leveraging ArcFace, CLIP, and Q-Former with decoupled cross-attention, allows for flexible and modular injection of identity features without modifying the pre-trained diffusion U-Net.
- Alignment Learning effectively disentangles ID-relevant features from ID-irrelevant semantics by forcing consistent noise predictions across varied references of the same identity, significantly improving robustness and consistency.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.