Boosting Fine-Grained Visual Anomaly Detection with Coarse-Knowledge-Aware Adversarial Learning

TL;DR

This work tackles fine-grained visual anomaly detection by exploiting a small coarse anomaly dataset to overcome the limitations of purely unsupervised reconstruction-based methods. The authors introduce CKAAD, an energy-based, coarse-knowledge-aware adversarial framework that aligns reconstructed feature distributions with normal features, using image-level and patch-level discriminators to improve detection and localization. Theoretical guarantees accompany practical training and testing procedures, enabling effective use of incomplete anomaly information. Empirical results on four medical and two industrial datasets demonstrate consistent improvements in both detection and precise localization over strong baselines and several weakly supervised variants. This approach offers a practical pathway to enhanced anomaly monitoring in real-world safety-critical domains where only limited anomaly examples are available.

Abstract

Many unsupervised visual anomaly detection methods train an auto-encoder to reconstruct normal samples and then leverage the reconstruction error map to detect and localize the anomalies. However, due to the powerful modeling and generalization ability of neural networks, some anomalies can also be well reconstructed, resulting in unsatisfactory detection and localization accuracy. In this paper, a small coarsely-labeled anomaly dataset is first collected. Then, a coarse-knowledge-aware adversarial learning method is developed to align the distribution of reconstructed features with that of normal features. The alignment can effectively suppress the auto-encoder's reconstruction ability on anomalies and thus improve the detection accuracy. Considering that anomalies often only occupy very small areas in anomalous images, a patch-level adversarial learning strategy is further developed. Although no patch-level anomalous information is available, we rigorously prove that by simply viewing any patch features from anomalous images as anomalies, the proposed knowledge-aware method can also align the distribution of reconstructed patch features with the normal ones. Experimental results on four medical datasets and two industrial datasets demonstrate the effectiveness of our method in improving the detection and localization performance.

Figures (7)

• Figure 1: Framework of our proposed CKAAD. With the normal and anomalous feature maps extracted from pre-trained ResNet as well as the generated feature maps output from the auto-encoder, the energy-based discriminator $D^\ell$ is trained to distinguish between normal, anomaly, and generated patch features output from the $l$-th layer of ResNet. The auto-encoder is then incentivized to always output normal features regardless of the input types.
• Figure 2: Histograms of performance (Average AUC and F1 of 5 random experiments) with different number ($k$) of labeled anomaly classes on ISIC2018 under $r_l=5\%$.
• Figure 3: Visualization of detection results. Images, anomaly maps of Recon, ReconSub, and our method are shown in the first column and last three columns respectively. The second column in (b) circles anomalous areas by red lines.
• Figure 4: Visualization of elastic transformation of images.
• Figure 5: The two rows separately exhibit anomaly score distribution on the test set trained without/with $5\%$ anomalies.

Theorems & Definitions (10)

• Theorem 1
• proof
• Theorem 2
• proof
• Lemma 1
• proof
• Theorem 1
• proof
• Theorem 2
• proof