Reproduce “VADER: Adaptive Debiasing for Hallucination Mitigation in Video Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dong Xing, Jiaxin Chen, Hang Yang, Peixun Liu, Qiushi Yang, Yuqing Wang
The VADER framework introduces a training-free approach to mitigate hallucinations in Video Large Language Models by adaptively reallocating visual attention and selectively erasing key evidence for robust prior-biased signal construction. It improved accuracy on the EventHallusion benchmark by up to 9.03 points (from 63.57% to 72.60% on LLaVA-Video-7B) while maintaining general video understanding performance.
Key insights
- Adaptive, input-conditioned visual interventions are crucial for effectively debiasing VideoLLMs, as the optimal intervention layers and strengths vary dynamically with each video-question input.
- Generating a reliable prior-biased branch for contrastive decoding in video requires selective, per-frame internal evidence erasure that removes high-importance visual tokens while preserving low-importance context to prevent models from compensating with redundant information.
- The combination of proactively enhancing visual evidence (VFR) and reactively exposing prior biases (SEE) through a dual-branch contrastive framework offers a highly complementary and effective strategy for hallucination mitigation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.