Reproduce “Mind the Way You Select Negative Texts: Pursuing the Distance Consistency in OOD Detection with VLMs” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zhikang Xu, Qianqian Xu, Zitai Wang, Cong Hua, Sicong Li, Zhiyong Yang, Qingming Huang

Researchers from the Chinese Academy of Sciences developed InterNeg, a framework for Out-of-Distribution (OOD) detection in Vision-Language Models (VLMs) that resolves a conflict between intra-modal OOD detection metrics and the VLM's inter-modal optimization objective. The method achieves state-of-the-art zero-shot OOD detection, improving AUROC by 0.77% and reducing FPR95 by 3.47% on ImageNet-1K benchmarks while mitigating ID misclassification.

Key insights

  • A fundamental inconsistency exists in many VLM-based OOD detection methods where intra-modal distance reliance conflicts with the inter-modal training objective of VLMs like CLIP.
  • Explicitly enforcing inter-modal distance consistency, both in pre-selecting and dynamically generating negative texts, is crucial for improving OOD detection robustness and reducing ID misclassification.
  • ID image proxies, generated from a small number of ID images, can effectively anchor inter-modal distance calculations for robust negative text selection and filtering in a zero-shot OOD detection setting.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.