Reproduce “Multimodal RewardBench 2: Evaluating Omni Reward Models for Interleaved Text and Image” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yushi Hu, Reyhane Askari-Hemmat, Melissa Hall, Emily Dinan, Luke Zettlemoyer, Marjan Ghazvininejad
Multimodal RewardBench 2 (MMRB2) introduces the first comprehensive benchmark for evaluating reward models on multimodal understanding and interleaved text and image generation. It reveals that frontier MLLM judges like Gemini 3 Pro achieve the highest human agreement (up to 80%) but still struggle with nuanced distinctions and exhibit biases in reasoning tasks, while demonstrating a strong correlation between benchmark performance and downstream generative AI task success.
Key insights
- The benchmark successfully highlights current limitations of MLLM-as-a-judge, revealing that even frontier models like Gemini 3 Pro, while leading, do not fully match human judgment.
- MLLM judges struggle with fine-grained differentiation between similar outputs from the same model and exhibit a bias towards image-containing responses in multimodal reasoning tasks.
- Performance on MMRB2 shows a strong correlation (Pearson's r > 0.8) with success in various downstream generative AI tasks, affirming its practical value for model selection and optimization.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.