Reproduce “NexViTAD: Few-shot Unsupervised Cross-Domain Defect Detection via Vision Foundation Models and Multi-Task Learning” on OpenResearch

About this paper

Tianwei Mu, Feiyu Duan, Bo Zhou, Dan Xue, Manhong Huang

The NEXVITAD framework addresses few-shot unsupervised cross-domain defect detection by integrating vision foundation models with multi-task learning and an unsupervised inference pipeline. It achieved state-of-the-art performance on the MVTec AD dataset, showing 97.5% AUC, 70.4% AP, and 95.2% PRO for pixel-level anomaly localization across different industrial domains.

Key insights

• The synergistic combination of hierarchical (Hiera) and semantic (DINOv2) vision foundation models via feature interleaving provides highly robust and transferable feature representations resilient to domain shifts.
• Multi-task learning with separate decoder heads for source categories and pseudo-labeled target categories, enforced by perturbation consistency, effectively disentangles feature learning and facilitates unsupervised adaptation.
• Utilizing a memory bank with Sinkhorn K-means clustering during inference dynamically models normal feature distributions, which is crucial for achieving superior pixel-level anomaly localization in an unsupervised cross-domain setting.

Summary by alphaXiv.