Reproduce “UNO-Bench: A Unified Benchmark for Exploring the Compositional Law Between Uni-modal and Omni-modal in Omni Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Chen Chen, ZeYang Hu, Fengjiao Chen, Liya Ma, Jiaxing Liu, Xiaoyu Li, Ziwen Wang, Xuezhi Cao and 1 more

Meituan researchers developed UNO-Bench, a unified benchmark for evaluating both uni-modal and omni-modal understanding in AI models, specifically catering to the Chinese linguistic context with high-quality, human-curated data. This work also empirically validates a "Compositional Law" that quantitatively describes the power-law relationship between a model's uni-modal and omni-modal performance.

Key insights

  • Omni-modal capability follows a "Compositional Law," expressible as P_Omni ≈ 1.0332 (P_A × P_V)^2.1918 + 0.2422, where the super-linear exponent (α ≈ 2.19) indicates an accelerating performance gain as uni-modal abilities improve.
  • This power-law explains a "short-board effect" for weaker models and "emergent abilities" for stronger models, where uni-modal improvements lead to disproportionately larger omni-modal gains.
  • Complex, multi-step reasoning (as measured by MO questions) remains a significant challenge for all current omni models, representing a universal performance bottleneck.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.