Reproduce “VERHallu: Evaluating and Mitigating Event Relation Hallucination in Video Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zefan Zhang, Kehua Zhu, Shijie Jiang, Hongyuan Lu, Shengkai Sun, Tian Bai
Researchers from Jilin University and collaborators developed VERHallu, a benchmark specifically designed to evaluate and expose event relation hallucination in Video Large Language Models (VideoLLMs) through counterintuitive video scenarios. They also introduced Key-Frame Propagating (KFP), a training-free strategy that consistently improves models' ability to understand complex event relations by enhancing visual feature attention.
Key insights
- Current state-of-the-art VideoLLMs exhibit severe event relation hallucination, performing poorly on complex causal, temporal, and subevent reasoning tasks, often close to or worse than random chance.
- Models frequently rely on vision-language or pure language priors rather than grounded visual evidence, with "vision&language bias" being more pronounced in QA tasks.
- Model size alone does not resolve event relation hallucination, as larger and closed-source models still struggle, indicating deeper architectural or training data limitations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.