SSVP: Synergistic Semantic-Visual Prompting for Industrial Zero-Shot Anomaly Detection

TL;DR

Synergistic Semantic-Visual Prompting (SSVP) is proposed, that efficiently fuses diverse visual encodings to elevate model's fine-grained perception and deeply integrates DINOv3's multi-scale structural priors into the CLIP semantic space.

Abstract

Zero-Shot Anomaly Detection (ZSAD) leverages Vision-Language Models (VLMs) to enable supervision-free industrial inspection. However, existing ZSAD paradigms are constrained by single visual backbones, which struggle to balance global semantic generalization with fine-grained structural discriminability. To bridge this gap, we propose Synergistic Semantic-Visual Prompting (SSVP), that efficiently fuses diverse visual encodings to elevate model's fine-grained perception. Specifically, SSVP introduces the Hierarchical Semantic-Visual Synergy (HSVS) mechanism, which deeply integrates DINOv3's multi-scale structural priors into the CLIP semantic space. Subsequently, the Vision-Conditioned Prompt Generator (VCPG) employs cross-modal attention to guide dynamic prompt generation, enabling linguistic queries to precisely anchor to specific anomaly patterns. Furthermore, to address the discrepancy between global scoring and local evidence, the Visual-Text Anomaly Mapper (VTAM) establishes a dual-gated calibration paradigm. Extensive evaluations on seven industrial benchmarks validate the robustness of our method; SSVP achieves state-of-the-art performance with 93.0\% Image-AUROC and 92.2\% Pixel-AUROC on MVTec-AD, significantly outperforming existing zero-shot approaches.

Figures (8)

• Figure 1: Conceptual comparison of ZSAD paradigms: unlike traditional shallow fusion methods, SSVP establishes a deep synergistic flow for adaptive prompt generation. The red dashed lines highlight the distinction between our SSVP method and traditional methods.
• Figure 2: Framework of Synergistic Semantic-Visual Prompting (SSVP). Three core modules: HSVS aligns semantic and structural features; VCPG generates vision-conditioned prompts; VTAM refines scores via local-global interaction.
• Figure 3: Qualitative comparison with SOTA methods. Visualization of anomaly heatmaps shows that SSVP produces significantly stronger responses on anomalous regions and achieves clearer foreground-background separation compared to standard ZSAD methods.
• Figure 4: Visual comparison of anomaly localization between Baseline and SSVP. The columns display: (a) Original input images; (b) The generated anomaly score maps; (c) Feature maps processed by threshold filtering, where ground-truth areas are outlined by green contours; and (d) The ground-truth masks, indicated by red regions.
• Figure 5: Visualization of ZSAD results across multiple datasets. We visualize sample results from BTAD, DAGM, DTD-Synthetic, and KSDD2. The column layout is identical to Figure \ref{['fig:vis_baseline_comp']}: Original images; Anomaly score maps; Thresholded features with GT contours; and GT red masks.