Reproduce “MECD+: Unlocking Event-Level Causal Graph Discovery for Video Reasoning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Tieyuan Chen, Huabin Liu, Yi Wang, Yihang Chen, Tianyao He, Chaofan Gan, Huanyu He, Weiyao Lin

Researchers at Shanghai Jiao Tong University introduced the Multi-Event Causal Discovery (MECD) task and developed the Video Granger Causality Model (VGCM) to uncover comprehensive event-level causal graphs in long videos. VGCM achieved 71.28% accuracy and 3.94 Structural Hamming Distance on the MECD dataset, surpassing leading LLMs and VideoLLMs while demonstrating 3-6 times faster inference and significantly enhancing performance in video question answering and event prediction.

Key insights

  • The principles of Granger Causality can be effectively adapted to multi-modal video data to discover complex event-level causal graphs when combined with sophisticated causal inference techniques.
  • Explicitly designed mechanisms to address causality confounding and illusory causality are essential for reliable causal discovery in videos, as general-purpose large language models struggle with these issues.
  • A specialized causal reasoning model significantly outperforms general-purpose LLMs and VideoLLMs in event-level causal graph discovery, highlighting the need for tailored architectures for complex causal inference tasks.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.