Reproduce “VT-ADL: A Vision Transformer Network for Image Anomaly Detection and Localization” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Pankaj Mishra, Riccardo Verk, Daniele Fornasier, Claudio Piciarelli, Gian Luca Foresti

VT-ADL, a Vision Transformer-based network, addresses semi-supervised image anomaly detection and pixel-level localization by integrating a ViT encoder, a decoder for reconstruction, and a Gaussian Mixture Density Network for latent space modeling. The approach achieves a mean AUC of 0.984 on MNIST, a mean PRO score of 0.807 on MVTec, and strong performance on a new industrial dataset, BTAD.

Key insights

  • Vision Transformers, when combined with reconstruction and latent space density modeling, are effective for both image-level anomaly detection and precise pixel-level localization.
  • Modeling the distribution of transformer-encoded features using a GMDN allows for robust anomaly scoring at a patch level, which can be upsampled to generate accurate localization heatmaps.
  • Adding Gaussian noise to transformer-embedded features before the GMDN significantly improves generalization and anomaly localization performance by acting as an effective regularization technique.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.