Reproduce “AUPIMO: Redefining Visual Anomaly Detection Benchmarks with High Speed and Low Tolerance” on OpenResearch

About this paper

Joao P. C. Bertoldo, Dick Ameln, Ashwin Vaidya, Samet Akçay

Researchers from Mines Paris, PSL University, and Intel introduced AUPIMO, a novel evaluation metric for unsupervised anomaly localization that redefines benchmarks by imposing a strict low false positive tolerance and bias-free validation. This metric revealed that existing datasets are not "near-solved" and differentiates model performance more effectively than traditional metrics like AUROC and AUPRO.

Key insights

• AUPIMO's bias-free validation and strict low false positive tolerance provide a more rigorous and practically relevant assessment of anomaly localization models.
• The metric reveals that benchmark datasets, previously considered 'near-solved' by other metrics, still present significant challenges for state-of-the-art models.
• The ability to generate per-image scores facilitates deeper performance diagnostics, enabling researchers to identify specific challenging instances and perform robust model comparisons using statistical tests.

Summary by alphaXiv.