Reproduce “Seeing What Matters: Visual Preference Policy Optimization for Visual Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ziqi Ni, Yuanzhi Liang, Rui Li, Yi Zhou, Haibin Huang, Chi Zhang, Xuelong Li
Visual Preference Policy Optimization (ViPO) is introduced as an enhanced Group Relative Policy Optimization (GRPO) framework that provides structured, pixel-level advantages for visual content generation. By reformulating the advantage representation using a Perceptual Structuring Module, ViPO achieves superior perceptual fidelity, realism, and alignment with human preferences in both image and video synthesis, outperforming scalar-reward GRPO across various benchmarks.
Key insights
- Transforming coarse scalar advantages into structured, pixel-level, and temporally aware advantages provides a more granular and informative learning signal for visual generative models.
- Integrating perceptual insights from pretrained vision backbones into the advantage assignment process effectively directs optimization pressure towards perceptually important regions.
- The modular and lightweight design of the PSM allows for architecture-agnostic integration and maintains compatibility with existing GRPO pipelines with minimal computational overhead.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.