One Language-Free Foundation Model Is Enough for Universal Vision Anomaly Detection

TL;DR

UniADet presents a language-free foundation-model framework for universal vision anomaly detection, showing that language prompts are unnecessary for zero-/few-shot AD. By decoupling global image-level classification and local pixel-level segmentation with independent weights across multi-scale features, UniADet mitigates manifold conflicts and achieves state-of-the-art results on 14 industrial/medical benchmarks. A few-shot extension uses a memory bank of normal patches to boost performance, while Class-Aware Augmentations enhance robustness. The approach delivers highly competitive or superior results with far fewer learnable parameters and faster inference, demonstrating strong cross-domain generalization and practical efficiency for open-world anomaly detection.

Abstract

Universal visual anomaly detection (AD) aims to identify anomaly images and segment anomaly regions towards open and dynamic scenarios, following zero- and few-shot paradigms without any dataset-specific fine-tuning. We have witnessed significant progress in widely use of visual-language foundational models in recent approaches. However, current methods often struggle with complex prompt engineering, elaborate adaptation modules, and challenging training strategies, ultimately limiting their flexibility and generality. To address these issues, this paper rethinks the fundamental mechanism behind visual-language models for AD and presents an embarrassingly simple, general, and effective framework for Universal vision Anomaly Detection (UniADet). Specifically, we first find language encoder is used to derive decision weights for anomaly classification and segmentation, and then demonstrate that it is unnecessary for universal AD. Second, we propose an embarrassingly simple method to completely decouple classification and segmentation, and decouple cross-level features, i.e., learning independent weights for different tasks and hierarchical features. UniADet is highly simple (learning only decoupled weights), parameter-efficient (only 0.002M learnable parameters), general (adapting a variety of foundation models), and effective (surpassing state-of-the-art zero-/few-shot by a large margin and even full-shot AD methods for the first time) on 14 real-world AD benchmarks covering both industrial and medical domains. We will make the code and model of UniADet available at https://github.com/gaobb/UniADet.

Figures (5)

• Figure 1: Comparisons of language-dependent zero-shot AD and our language-free universal AD (UniADet). UniADet is simple (learning only task-related weights), parameter-efficient (about 0.001M learnable parameters), general (adapting a variety of foundation models), and effective (surpassing state-of-the-art zero-/few-shot and even full-shot AD methods).
• Figure 2: The language-free UniADet framework. UniADet decouples global image anomaly classification and local patch segmentation so that their weights are learned independently across hierarchical features. Once trained, it can identify any anomaly images and segment anomaly regions, providing only few-shot and even zero-shot normal images.
• Figure 3: t-SNE visualization of CLIP ViT-L/14@336px features on MVTec Test Set (Hazelnuts). (a) t-SNE embeddings of global image tokens ($\vec{x}^q$) and local patch tokens ($F^q$) extracted from the $24$-th block. The visualization clearly shows a significant disparity between the normal/anomaly distributions of image (global) tokens and patch (local) tokens. (b) t-SNE embeddings of local patch tokens extracted from the $6$-th and $24$-th blocks. The normal and anomaly feature distributions across these two distinct hierarchical layers are also substantially different.
• Figure 4: Qualitative comparisons of UniADet using foundation DINOv3 (ViT-L/16) on industrial and medical domains. Best viewed in color and zoom.
• Figure 5: t-SNE visualization of features extracted diverse foundation models, e.g., DINOv2R ViT-L/14, and DINOv3 ViT-L/16 on MVTec Test Set (Hazelnuts).