Reproduce “Compositional Text-to-Image Generation Via Region-aware Bimodal Direct Preference Optimization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhuohan Liu, Wujian Peng, Yitong Chen, Zuxuan Wu
A framework called BiDPO is introduced, extending Direct Preference Optimization with bimodal and region-aware capabilities to improve compositional text-to-image generation. This approach, supported by an automatically constructed BiComp preference dataset, boosts attribute binding accuracy on T2I-CompBench by up to 20.4% and raises overall GenEval scores by 0.09 compared to baseline models.
Key insights
- Direct Preference Optimization (DPO) can be effectively adapted to enhance compositional capabilities in text-to-image models through bimodal preference alignment.
- Explicitly incorporating both textual and visual preference signals, combined with region-specific guidance, is crucial for improving fine-grained compositional accuracy.
- A carefully curated, automatically generated preference dataset with region annotations is vital for training models to understand complex compositional instructions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.