Reproduce “AF-CLIP: Zero-Shot Anomaly Detection via Anomaly-Focused CLIP Adaptation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Qingqing Fang, Wenxi Lv, Qinliang Su

Researchers at Sun Yat-sen University developed AF-CLIP, a framework that adapts the CLIP vision-language model to focus on local anomalies, enhancing zero-shot and few-shot visual anomaly detection. This approach improved image-level AUROC to 92.9% and pixel-level AUROC to 92.3% on MVTec AD, outperforming existing methods.

Key insights

  • Explicitly adapting the visual encoder of pre-trained VLMs, rather than solely relying on textual prompt learning, is crucial for improving fine-grained anomaly detection performance and localization accuracy.
  • Multi-scale spatial information aggregation combined with an attention-based adapter enables CLIP to effectively capture and focus on localized anomalies of varying sizes.
  • A dedicated patch alignment loss can successfully enforce more compact and separable feature clusters for normal and anomalous patches, enhancing the model's ability to distinguish defects.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.