Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CNC: Cross-modal Normality Constraint for Unsupervised Multi-class Anomaly Detection

Xiaolei Wang, Xiaoyang Wang, Huihui Bai, Eng Gee Lim, Jimin Xiao

TL;DR

This work addresses the challenge of over-generalization in unsupervised multi-class anomaly detection by introducing a cross-modal normality constraint (CNC) that aligns decoded visual features with a universal textual representation of normality via category-agnostic prompts. It further mitigates interference among diverse patch patterns with a gated mixture-of-experts (MoE) module, built on a CLIP-ViT backbone to enable cross-modal learning from text prompts. The proposed framework, including cross-modal feature distillation and normality-guided decoding, achieves competitive or state-of-the-art results on MVTec AD and VisA, demonstrating improved anomaly localization and image-level detection in multi-class settings. Collectively, CNC and MoE enhance robustness to multi-class diversity while preserving normality generalization, with practical implications for scalable industrial defect detection.

Abstract

Existing unsupervised distillation-based methods rely on the differences between encoded and decoded features to locate abnormal regions in test images. However, the decoder trained only on normal samples still reconstructs abnormal patch features well, degrading performance. This issue is particularly pronounced in unsupervised multi-class anomaly detection tasks. We attribute this behavior to over-generalization(OG) of decoder: the significantly increasing diversity of patch patterns in multi-class training enhances the model generalization on normal patches, but also inadvertently broadens its generalization to abnormal patches. To mitigate OG, we propose a novel approach that leverages class-agnostic learnable prompts to capture common textual normality across various visual patterns, and then apply them to guide the decoded features towards a normal textual representation, suppressing over-generalization of the decoder on abnormal patterns. To further improve performance, we also introduce a gated mixture-of-experts module to specialize in handling diverse patch patterns and reduce mutual interference between them in multi-class training. Our method achieves competitive performance on the MVTec AD and VisA datasets, demonstrating its effectiveness.

CNC: Cross-modal Normality Constraint for Unsupervised Multi-class Anomaly Detection

TL;DR

This work addresses the challenge of over-generalization in unsupervised multi-class anomaly detection by introducing a cross-modal normality constraint (CNC) that aligns decoded visual features with a universal textual representation of normality via category-agnostic prompts. It further mitigates interference among diverse patch patterns with a gated mixture-of-experts (MoE) module, built on a CLIP-ViT backbone to enable cross-modal learning from text prompts. The proposed framework, including cross-modal feature distillation and normality-guided decoding, achieves competitive or state-of-the-art results on MVTec AD and VisA, demonstrating improved anomaly localization and image-level detection in multi-class settings. Collectively, CNC and MoE enhance robustness to multi-class diversity while preserving normality generalization, with practical implications for scalable industrial defect detection.

Abstract

Existing unsupervised distillation-based methods rely on the differences between encoded and decoded features to locate abnormal regions in test images. However, the decoder trained only on normal samples still reconstructs abnormal patch features well, degrading performance. This issue is particularly pronounced in unsupervised multi-class anomaly detection tasks. We attribute this behavior to over-generalization(OG) of decoder: the significantly increasing diversity of patch patterns in multi-class training enhances the model generalization on normal patches, but also inadvertently broadens its generalization to abnormal patches. To mitigate OG, we propose a novel approach that leverages class-agnostic learnable prompts to capture common textual normality across various visual patterns, and then apply them to guide the decoded features towards a normal textual representation, suppressing over-generalization of the decoder on abnormal patterns. To further improve performance, we also introduce a gated mixture-of-experts module to specialize in handling diverse patch patterns and reduce mutual interference between them in multi-class training. Our method achieves competitive performance on the MVTec AD and VisA datasets, demonstrating its effectiveness.
Paper Structure (34 sections, 17 equations, 4 figures, 4 tables)

This paper contains 34 sections, 17 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Anomaly Detection
  4. Reconstruction-based Method
  5. Feature-embedding-based Method
  6. Augmentation-based Method
  7. Preliminaries
  8. CLIP
  9. Prompt Learning
  10. Methodology
  11. Overview
  12. Visual Distillation Framework
  13. Cross-modal Normality Constraint
  14. Learning Cross-modal Normality
  15. Feature Distillation with Normality Promotion
  16. ...and 19 more sections

Figures (4)

  • Figure 1: (A) and (B) show the correspondence between visual and text modality and the motivation of CNC, respectively. (C) shows a comprehensive performance comparison with previous SOTA methods that only learn sample normality on the visual modality on MVTec AD dataset. (D) gives some visualization results of cross-modal constraint on MVTec AD dataset.
  • Figure 2: Overview of the proposed cross-modal normality distillation framework. Additionally, details of feature-level normality promotion, multi-layer fusion, and gated mixture-of-experts are illustrated in this graph.
  • Figure 3: Visualization for detection results of UniAD and our method on MVTec AD and VisA datasets.
  • Figure 4: Choices on four pre-trained teacher network (encoder) with four different resolutions. The vertical axis represents the average value of I-AUROC/P-AUROC/AUPRO/I-mAP/P-mAP. ViT-L/14* denotes pre-trained CLIP model, VIT-L/14@336px.