Reproduce “CLEVRER: CoLlision Events for Video REpresentation and Reasoning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kexin Yi, Chuang Gan, Yunzhu Li, Pushmeet Kohli, Jiajun Wu, Antonio Torralba, Joshua B. Tenenbaum
CLEVRER introduces a synthetic video dataset designed to assess AI models' temporal and causal reasoning abilities in explaining, predicting, and counterfactually reasoning about collision events. The research also presents the Neuro-Symbolic Dynamic Reasoning (NS-DR) model, which achieves substantial performance improvements over existing baselines across these complex reasoning tasks.
Key insights
- State-of-the-art visual reasoning models, while competent at descriptive tasks, fundamentally lack the ability to model dynamics and infer causal relationships, particularly for unobserved or counterfactual situations.
- Object-centric representations are crucial for robust video reasoning, as evidenced by improved performance when models process individual objects and their attributes.
- A neuro-symbolic architecture that combines neural perception and dynamics prediction with symbolic logic provides an effective framework for addressing complex causal video understanding challenges.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.