Sub-Image Anomaly Detection with Deep Pyramid Correspondences

TL;DR

This work addresses sub-image anomaly detection under a normal-only training regime by introducing SPADE, a two-stage, correspondence-based approach that leverages a multi-resolution deep feature pyramid to locate anomalies. By retrieving $K$ nearest normal images and building a pixel-level gallery of deep features, SPADE computes per-pixel anomaly scores via dense correspondences, obviating extensive training. The method achieves state-of-the-art results on the MVTec and Shanghai Tech Campus datasets for both image-level anomaly detection and pixel-level localization, while remaining fast and easy to deploy. The combination of pre-trained ImageNet features, kNN-based normal retrieval, and multi-scale feature matching yields robust, explainable localization suitable for industrial and surveillance applications.

Abstract

Nearest neighbor (kNN) methods utilizing deep pre-trained features exhibit very strong anomaly detection performance when applied to entire images. A limitation of kNN methods is the lack of segmentation map describing where the anomaly lies inside the image. In this work we present a novel anomaly segmentation approach based on alignment between an anomalous image and a constant number of the similar normal images. Our method, Semantic Pyramid Anomaly Detection (SPADE) uses correspondences based on a multi-resolution feature pyramid. SPADE is shown to achieve state-of-the-art performance on unsupervised anomaly detection and localization while requiring virtually no training time.

Figures (3)

• Figure 1: An evaluation of SPADE on detecting anomalies between flowers with or without insects (taken from one category of 102 Category Flower Dataset nilsback2008automated) and bird varieties (taken from Caltech-UCSD Birds 200) welinder2010caltech. (left to right) i) An anomalous image ii) The retrieved top normal neighbor image iii) The mask detected by SPADE iv) The predicted anomalous image pixels. SPADE was able to detect the insect on the anomalous flower (top), the white colors of the anomalous albatross (center) and the red spot on the anomalous bird (bottom).
• Figure 2: (left to right) i) An anomalous image ii) The retrieved top normal neighbor image iii) The mask detected by SPADE iv) The predicted anomalous image pixels. We can see how in this example, SPADE detects the anomalous image region by finding the correspondence with the nearest-neighbor image. The anomalous parts did not have correspondences in the normal image and were therefore detected.
• Figure 3: (top rows) Anomaly detection on images from the Cable and Grid categories of the MVTec dataset (bottom) Detecting bike anomaly on the STC dataset.