Reproduce “DominoBench: Evaluating Multi-Step Physical Reasoning from Still Images” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Emilio Ferrara

DominoBench tests whether vision models can predict outcomes in paired, simulated two-dimensional chain reactions that differ near a physical threshold. Its hint ladder progresses from an image-only question to a supplied calculation and its inputs. Across 854 items, four Claude models reach 82–86% deciding-person accuracy with the calculation, versus 57–61% unaided; this tests supplied-information use, not unaided derivation.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.