Reproduce “Ctrl-U: Robust Conditional Image Generation via Uncertainty-aware Reward Modeling” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Guiyu Zhang, Huan-ang Gao, Zijian Jiang, Hao Zhao, Zhedong Zheng

Researchers from the University of Macau and Tsinghua University developed Ctrl-U, an uncertainty-aware reward modeling framework for robust conditional image generation. Ctrl-U quantifies and leverages the uncertainty in pre-trained reward models when evaluating newly generated images, adaptively rectifying feedback to enhance both visual fidelity and precise semantic alignment with user conditions, achieving up to 44.42% mIoU improvement on COCO-Stuff segmentation and an 18.14% FID reduction compared to ControlNet++.

Key insights

  • Pre-trained reward models exhibit 'cognitive uncertainty' when evaluating diverse, out-of-distribution images generated during the diffusion process, leading to unreliable feedback.
  • This uncertainty can be effectively quantified by observing the discrepancy in condition extraction from two slightly perturbed generations (via different noise timesteps) without needing additional model parameters.
  • Dynamically rectifying reward signals based on their estimated uncertainty during training significantly stabilizes the learning process and improves the robustness and accuracy of conditional image generation.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.