Reproduce “VideoPhy: Evaluating Physical Commonsense for Video Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hritik Bansal, Zongyu Lin, Tianyi Xie, Zeshun Zong, Michal Yarom, Yonatan Bitton, Chenfanfu Jiang, Yizhou Sun and 2 more
The VIDEOPHY benchmark systematically evaluates the physical plausibility of videos generated by text-to-video (T2V) models, revealing that current state-of-the-art models exhibit severe deficiencies in physical commonsense, with the best model achieving less than 40% combined semantic and physical accuracy. The study also introduces a finetuned automatic evaluator, VIDEOCON-PHYSICS, to scale this assessment.
Key insights
- Current T2V models demonstrate a severe lack of physical commonsense, consistently performing poorly on physically challenging interactions like solid-solid dynamics.
- Achieving high semantic adherence in video generation does not guarantee physical plausibility, indicating a distinct evaluation dimension that models struggle with.
- The `VIDEOCON-PHYSICS` auto-evaluator effectively assesses physical commonsense, demonstrating strong agreement with human judgments and generalizing well to unseen prompts and generative models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.