Reproduce “BeautyGRPO: Aesthetic Alignment for Face Retouching via Dynamic Path Guidance and Fine-Grained Preference Modeling” on OpenResearch

About this paper

Jiachen Yang, Xianhui Lin, Yi Dong, Zebiao Zheng, Xing Liu, Hong Gu, Yanmei Fang

Researchers from Sun Yat-sen University and vivo BlueImage Lab developed BeautyGRPO, a reinforcement learning framework for face retouching that effectively aligns outputs with human aesthetic preferences while preserving high visual fidelity. This system leverages a new fine-grained preference dataset and a multi-dimensional reward model, combined with Dynamic Path Guidance (DPG) to stabilize stochastic exploration, resulting in superior aesthetic quality and naturalness in retouched images as confirmed by quantitative metrics and user studies.

Key insights

• Achieving true aesthetic alignment in face retouching requires moving beyond pixel-level supervision to models that can actively learn from and respond to nuanced human preferences through reward signals.
• The fidelity-exploration conflict in SDE-based online RL can be resolved by dynamically guiding stochastic trajectories with high-quality anchors, allowing for effective exploration without sacrificing visual stability.
• A dedicated, fine-grained reward model, trained on multi-dimensional human preference data, is crucial for providing accurate and perceptually relevant feedback to guide reinforcement learning in subtle aesthetic tasks.

Summary by alphaXiv.