Mitigating Spurious Negative Pairs for Robust Industrial Anomaly Detection

TL;DR

One-class anomaly detection often lacks anomaly examples, making detectors vulnerable to adversarial perturbations. The authors propose COBRA, combining distribution-aware pseudo-anomaly generation with an anomaly-aware contrastive loss that uses opposite pairs to maximize the margin between normal and anomaly distributions in embedding space under adversarial perturbations bounded by $\epsilon\u007f$. A threshold $lambda$ computed via a k-class classifier and Gaussian Mixture Model on embeddings filters pseudo-anomalies, and the training optimizes a min–max objective with a binary head for scoring. Empirically, COBRA yields up to $26.1\%$ improvements in robust AUROC across multiple datasets and attacks, without requiring external anomaly data, while maintaining competitive clean performance and demonstrating broad applicability to real-world anomaly detection tasks.

Abstract

Despite significant progress in Anomaly Detection (AD), the robustness of existing detection methods against adversarial attacks remains a challenge, compromising their reliability in critical real-world applications such as autonomous driving. This issue primarily arises from the AD setup, which assumes that training data is limited to a group of unlabeled normal samples, making the detectors vulnerable to adversarial anomaly samples during testing. Additionally, implementing adversarial training as a safeguard encounters difficulties, such as formulating an effective objective function without access to labels. An ideal objective function for adversarial training in AD should promote strong perturbations both within and between the normal and anomaly groups to maximize margin between normal and anomaly distribution. To address these issues, we first propose crafting a pseudo-anomaly group derived from normal group samples. Then, we demonstrate that adversarial training with contrastive loss could serve as an ideal objective function, as it creates both inter- and intra-group perturbations. However, we notice that spurious negative pairs compromise the conventional contrastive loss to achieve robust AD. Spurious negative pairs are those that should be closely mapped but are erroneously separated. These pairs introduce noise and misguide the direction of inter-group adversarial perturbations. To overcome the effect of spurious negative pairs, we define opposite pairs and adversarially pull them apart to strengthen inter-group perturbations. Experimental results demonstrate our superior performance in both clean and adversarial scenarios, with a 26.1% improvement in robust detection across various challenging benchmark datasets. The implementation of our work is available at: https://github.com/rohban-lab/COBRA.

Figures (5)

• Figure 1: ①: Given a training batch that includes normal group samples $A$ and $B$, we create a anomaly group using our proposed transformation ${\Upsilon}_{\lambda}$. These samples are paired as opposite pairs (e.g., $A$ and $A^{\Upsilon}$) and subjected to $\tau_1$ and $\tau_2$ to form batches of positive pairs. Adversarial training is performed with a loss function combining $L_{\text{CLS}}$ and $L_{\text{COBRA}}$, where $L_{\text{COBRA}}$ treats adversarial examples as positive pairs for the corresponding sample. ②, ③: The illustrations demonstrate how $L_{\text{COBRA}}$ enhances adversarial training by explicitly increasing similarities within positive pairs and decreasing similarity for opposite pairs, thus creating strong inter- and intra-group perturbations. Targeting opposite pairs instead of all negatives diminishes the effect of spurious negative pairs (e.g., $(A_1, B_1)$), leading to stronger inter-group perturbations and enlarging the margin between distributions for normal and anomaly groups. A detailed algorithmic of COBRA is provided in \ref{['appendix_algorithm_model']}.
• Figure 2: The figure highlights the challenges CPAD and CSI face in generating inncorrect anomalies due to the absence of a threshold. Techniques such as FITYMI and Dream-OOD, which generate anomalies from the embedding space of a pretrained model, typically lead to a loss of pixel-level detail and show biases towards the dataset used for pre-training (e.g., ImageNet). Such biases decrease their effectiveness on datasets not seen during pre-training, such as medical imaging datasets like ISIC. In contrast, COBRA, by adapting to the distribution of the normal dataset, efficiently crafts informative anomalies in the pixel space and utilizes a thresholding method to filter out incorrect anomalies, all without the need for any additional datasets.
• Figure 3: UMAP visualization of features extracted by the encoder $f$, trained with various loss functions on the CIFAR-10 dataset, is presented in a one-class setup with the 'Automobile' class designated as the normal set. For this particular experiment, all elements except the loss function remain constant to ensure a fair comparison.
• Figure 4: This figure depicts COBRA's loss values and detection performance for each epoch of training on both clean data and data under PGD attack, demonstrating the stability of COBRA's loss function. The experiment was carried out in a one-class setup using the MVETEC-AD and FMNIST datasets.
• Figure 5: Computation cost