Reproduce “Where Did the Light Come From? Diagnosing Cast-Shadow Reasoning in VLMs and Borrowing the Missing Shadow Prior from Image Generation Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ryugo Morita, Shunsuke Kitada, Stanislav Frolov, Brian Bernhard Moser, Ko Watanabe, Riku Takahashi, Jinjia Zhou, Andreas Dengel
ShadowBench tests whether vision-language models can read visible cast-shadow direction, infer it when absent, and judge whether a shown shadow matches its scene. Starting from real photographs, the authors create matched composites and score direction choices by balanced accuracy. Across 19 open models, a generation-and-readback loop raises mean inference balanced accuracy from 35.6% to 58.9%, on this limited screen-space cast-shadow task.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.