Reproduce “Compositional 4D Dynamic Scenes Understanding with Physics Priors for Video Question Answering” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xingrui Wang, Wufei Ma, Angtian Wang, Shuo Chen, Adam Kortylewski, Alan Yuille
A new dataset, DynSuperCLEVR, is introduced to evaluate video question answering on 4D dynamic properties of 3D objects with enhanced realism. The NS-4DPhysics model, which integrates physics priors with explicit 4D scene representation and neural-symbolic reasoning, achieves 82.64% overall accuracy on this benchmark, outperforming large multimodal models by over 23%.
Key insights
- Large Multimodal Models and current VideoQA models fundamentally lack deep physical intuition and explicit 3D/4D understanding required for dynamic scene reasoning.
- Integrating explicit 4D scene representations with physics priors significantly improves a model's ability to understand dynamic properties and perform future and counterfactual reasoning.
- Neural-symbolic approaches, combining neural perception with symbolic inference on explicit physical states, offer a superior and more interpretable path for complex video understanding tasks involving physics.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.