Reproduce “ViPer: Visual Personalization of Generative Models via Individual Preference Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sogand Salehi, Mahdi Shafiei, Teresa Yeo, Roman Bachmann, Amir Zamir
ViPer, developed at EPFL, personalizes text-to-image generative models by learning individual visual preferences from free-form natural language comments. It modifies diffusion model guidance to align outputs with user tastes, achieving 86.1% top-1 user preference for personalized images over non-personalized ones and introducing an automated proxy metric that strongly correlates with human judgments.
Key insights
- Free-form natural language comments are highly effective and expressive for capturing diverse individual visual preferences, surpassing traditional sparse feedback methods.
- Modifying text embeddings and classifier-free guidance in diffusion models offers a flexible and efficient way to integrate personal preferences without retraining the base model.
- A synthetically trained proxy metric can reliably assess personalization alignment, enabling scalable and automated evaluation in personalized generative AI research.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.