Reproduce “No Place to Hide: Benchmarking Video Hallucination with Background-Controlled Pairs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Haojian Huang, Harold Haodong Chen, Meng Luo, Junjia Du, Shanqing Xu, Ziheng Chen, Yanxiang Huang, Yinchuan Li and 1 more
Researchers introduced VidPair-Halluc, a new benchmark using background-controlled video pairs to rigorously evaluate video hallucination in large video models (LVMs). Their PairFlow pipeline generates adversarial videos with highly similar backgrounds but distinct foreground semantics, revealing that current LVMs struggle significantly with fine-grained visual distinctions in these controlled settings.
Key insights
- Background-controlled adversarial video pairs present a significantly greater challenge to LVMs than text-only perturbations or videos with general background variations, exacerbating hallucination tendencies.
- Mainstream LVMs exhibit a trade-off between spatial sensitivity and hallucination control; qualitative analyses suggest models often prioritize broader scene context over subtle foreground semantic evidence.
- The `PairFlow` pipeline demonstrates the increasing viability and quality of generative AI for scalable construction of highly controlled, adversarial video datasets, with human verification rates improving to 83% with newer models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.