Reproduce “OARS: Process-Aware Online Alignment for Generative Real-World Image Super-Resolution” on OpenResearch

About this paper

Shijie Zhao, Xuanyu Zhang, Bin Chen, Weiqi Li, Qunliang Xing, Kexin Zhang, Yan Wang, Junlin Li and 3 more

Researchers from ByteDance, Peking University, and The Chinese University of Hong Kong developed OARS, a progressive online reinforcement learning framework for real-world image super-resolution, utilizing COMPASS, a process-aware reward model that dynamically balances fidelity and perceptual quality. The framework demonstrates state-of-the-art perceptual improvements, achieving a 47.62% user preference rate in subjective evaluations.

Key insights

• Evaluating the LR→SR transition as a process, rather than static outputs, with a process-aware reward (COMPASS) provides a more nuanced and effective signal for aligning generative SR models with human preferences.
• Dynamically balancing fidelity and perceptual gain based on the initial input quality (via the input-quality-adaptive mechanism in COMPASS) is crucial for robust real-world super-resolution.
• Progressive online RL with shallow LoRA optimization on a base model allows for better exploration and more stable learning in generative SR compared to direct fine-tuning of SFT weights, mitigating pseudo-diversity and reward hacking.

Summary by alphaXiv.