Reproduce “MDMR-Bench: A Multi-Dimensional Benchmark for Multi-Reference Image Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Bowen Zheng, Jintao Lin, Changheng Yi, Haoran Yang, Binxiao Huang, Chenyang Lei, Rui Liu, Xihui Liu
The authors introduce MDMR-Bench, a 2,390-case benchmark for multi-reference image generation, covering five task families and evaluating outputs with binary global and atomic checklist questions. Checklist judging has higher overall inter-judge agreement than scalar scoring (0.77 versus 0.60), but not for Quality. Under Gemini-3-Pro, GPT-Image-2 scores 88.8% Overall versus FLUX.2’s 51.3%; Overall equally averages task families, not individual cases.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.