Reproduce “PhysGame: Uncovering Physical Commonsense Violations in Gameplay Videos” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Meng Cao, Haoran Tang, Haoze Zhao, Hangyu Guo, Jiaheng Liu, Ge Zhang, Ruyang Liu, Qiang Sun and 2 more
This research introduces PhysGame, the first benchmark specifically designed to evaluate Video LLMs' ability to identify physical commonsense violations in gameplay videos. The paper also develops PhysInstruct and PhysDPO, specialized datasets for training, and presents PhysVLM, a Video LLM that achieves 80.3% accuracy on PhysGame, outperforming other open-source models and approaching proprietary model performance while maintaining general video understanding.
Key insights
- Specialized training using instruction-tuning (PhysInstruct) and preference optimization (PhysDPO) on physical commonsense data is crucial for enhancing Video LLMs' ability to detect physics violations.
- The ability to identify physical anomalies in gameplay videos provides a robust testbed for evaluating AI systems' understanding of fundamental physics principles.
- Improving physical commonsense reasoning in Video LLMs does not diminish their general video understanding capabilities, suggesting a complementary relationship.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.