Reproduce “Logical Closed Loop: Uncovering Object Hallucinations in Large Vision-Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Junfei Wu, Qiang Liu, Ding Wang, Jinghao Zhang, Shu Wu, Liang Wang, Tieniu Tan
The LogicCheckGPT framework addresses object hallucination in large vision-language models (LVLMs) by leveraging their internal logical consistency through a training-free, closed-loop question-answering mechanism. It led to over 30% accuracy improvement for mPLUG-Owl on the POPE adversarial setting and demonstrated superior performance compared to existing mitigation methods across multiple LVLMs.
Key insights
- LVLMs tend to exhibit logical consistency for objects that genuinely exist in an image but show inconsistency when probed about hallucinated objects.
- A quantifiable 'logical closed loop rate', calculated from an LVLM's ability to re-identify an object based on its previously described attributes, effectively indicates the likelihood of an object being hallucinated.
- Systematic, multi-turn natural language questioning can enable LVLMs to self-diagnose and correct factual errors in their outputs without requiring model retraining or access to internal parameters.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.