Reproduce “PSR: Scaling Multi-Subject Personalized Image Generation with Pairwise Subject-Consistency Rewards” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shulei Wang, Longhui Wei, Xin He, Jianbo Ouyang, Hui Lu, Zhou Zhao, Qi Tian
Researchers from Zhejiang University and Huawei Inc. developed a scalable framework for multi-subject personalized image generation, introducing a novel data generation pipeline and a reinforcement learning (RL) strategy with Pairwise Subject-Consistency Rewards (PSR). Their method achieved state-of-the-art performance on a new comprehensive benchmark, PSRBench, demonstrating superior subject consistency, semantic alignment, and aesthetic quality in generating images with multiple specified subjects.
Key insights
- Explicitly calculating consistency rewards at a pairwise subject level during RL fine-tuning significantly improves subject identity preservation in generated images, surpassing global image-level optimization.
- A data generation pipeline that synthesizes multi-subject data by re-contextualizing single-subject instances can overcome data scarcity and produce high-quality, consistent training examples.
- Employing a virtual temporal offset for positional encoding, instead of spatial offsets, enables a single-subject model to generalize effectively to multi-subject inputs without introducing unwanted spatial priors.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.