Reproduce “Evaluating Multiview Object Consistency in Humans and Image Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tyler Bonnen, Stephanie Fu, Yutong Bai, Thomas O'Connell, Yoni Friedman, Nancy Kanwisher, Joshua B. Tenenbaum, Alexei A. Efros
Researchers from UC Berkeley and MIT developed MOCHI, a benchmark to evaluate 3D object shape inference in humans and image models, revealing that current state-of-the-art vision models significantly underperform human observers, achieving 44% accuracy compared to human's 78%. The study suggests this gap stems from models lacking dynamic, time-dependent processing and attentional strategies employed by humans.
Key insights
- Human performance on 3D shape inference is robust and reliable, with an inverse correlation between accuracy and reaction time, and consistent gaze patterns, providing a rich target for model evaluation.
- Despite model scale increases, state-of-the-art vision models exhibit an unambiguous performance gap compared to humans, especially on tasks requiring robust 3D shape understanding of high-similarity objects.
- The human-model divergence is strongly linked to processing time and attention, suggesting humans resolve difficult inferences by allocating more time and employing specific attentional strategies that models do not replicate.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.