Towards Accurate Unified Anomaly Segmentation

TL;DR

UniAS achieves state-of-the-art anomaly segmentation performance, attaining 65.12/59.33 and 40.06/32.50 in pAP/DSC on the MVTec-AD and VisA datasets, respectively, surpassing previous methods significantly.

Abstract

Unsupervised anomaly detection (UAD) from images strives to model normal data distributions, creating discriminative representations to distinguish and precisely localize anomalies. Despite recent advancements in the efficient and unified one-for-all scheme, challenges persist in accurately segmenting anomalies for further monitoring. Moreover, this problem is obscured by the widely-used AUROC metric under imbalanced UAD settings. This motivates us to emphasize the significance of precise segmentation of anomaly pixels using pAP and DSC as metrics. To address the unsolved segmentation task, we introduce the Unified Anomaly Segmentation (UniAS). UniAS presents a multi-level hybrid pipeline that progressively enhances normal information from coarse to fine, incorporating a novel multi-granularity gated CNN (MGG-CNN) into Transformer layers to explicitly aggregate local details from different granularities. UniAS achieves state-of-the-art anomaly segmentation performance, attaining 65.12/59.33 and 40.06/32.50 in pAP/DSC on the MVTec-AD and VisA datasets, respectively, surpassing previous methods significantly. The codes are shared at https://github.com/Mwxinnn/UniAS.

Figures (5)

• Figure 1: Prediction examples (Left) of the MVTec dataset bergmann2019mvtec and failed cases (Right) of the SOTA model UniAD you2022unified. UniAD has high AUROC but relatively poor segmentation performance, improved by our UniAS.
• Figure 2: (Left) An example of MVTec-AD with a red line delineating anomalous GT bergmann2019mvtec, along with toy anomaly maps, segmentation predictions, and the corresponding metrics. Bad predictions can have high AUROC numbers. (Right) A real prediction example and corresponding metrics, showing the limitation of AUROC.
• Figure 3: (Left) The overview of UniAS. After extracting features, our multi-level hybrid decoder, composed of Transformer and MGG-CNN hybrid blocks, hierarchically reconstructs normal features from coarse to fine. Transformers and MGG-CNNs play complementary roles in global and local modeling. The SAR Query is incorporated to facilitate One-for-All segmentation. (Right) Illustration of the training loss and anomaly map generation during inference. By multiplying anomaly maps together, information from separate levels is aggregated, facilitating accurate anomaly segmentation.
• Figure 4: Qualitative results on MVTec-AD(left) and VisA(right). We visualize the anomaly maps and the generated masks. UniAS localizes anomaly precisely, exhibiting meaningful segmentation results.
• Figure 5: (Left) Examples of input images and corresponding anomaly maps from feature level 4 to level 1. (Right) Aggregated anomaly maps and the anomaly ground truth masks. Anomaly maps play complementary roles, detecting anomalous pixels from coarse to fine.