Reproduce “WorldExam: Benchmarking World Models from Apparent Appearance to Inherent Reactivity” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuxue Yang, Shuyao Shang, Jiahe Wang, Zitong Zhou, Liang Tan, Junhan Zeng, Ruizhi Li, Junyan Li and 8 more
Researchers from CASIA, CUHK, AMAP, and THU developed WorldExam, a diagnostic benchmark for video world models that assesses "inherent reactivity"—the capacity to infer and generate plausible scene-conditioned consequences beyond explicit instructions. Evaluations of 20 models revealed that current systems struggle with autonomous world reactions, even when demonstrating high visual quality or precise control adherence.
Key insights
- High visual quality or precise explicit control adherence does not guarantee a model's "inherent reactivity" or deep understanding of world dynamics.
- Current video world models exhibit distinct capability splits, with camera-driven models excelling in camera control, action-driven in subject control, and language-driven in interaction and goal-directed tasks.
- No single evaluated model consistently performed strongly across all diagnostic levels and tasks, indicating that current interfaces are complementary but lack holistic world comprehension.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.