HarmoniAD: Harmonizing Local Structures and Global Semantics for Anomaly Detection

TL;DR

HarmoniAD addresses the structure–semantics trade-off in industrial anomaly detection by introducing a frequency-guided dual-branch framework. A frozen CLIP encoder provides semantic features, which are separated in the frequency domain by a differentiable Soft Gate into high-frequency (FSAM) and low-frequency (GSCM) streams to jointly model texture/edges and global context. The architecture is strengthened by a fine-grained attention mechanism and a dynamic contextual module, coupled through a DMU, and trained with a multi-term loss to produce robust, pixel- and patch-level anomaly localization. Across MVTec-AD, VisA, and BTAD, HarmoniAD achieves state-of-the-art results with strong robustness and precision, while maintaining efficiency by freezing the CLIP backbone. The work highlights the value of explicit frequency-domain structure semantics separation for reliable anomaly detection and suggests future extensions to video settings.

Abstract

Anomaly detection is crucial in industrial product quality inspection. Failing to detect tiny defects often leads to serious consequences. Existing methods face a structure-semantics trade-off: structure-oriented models (such as frequency-based filters) are noise-sensitive, while semantics-oriented models (such as CLIP-based encoders) often miss fine details. To address this, we propose HarmoniAD, a frequency-guided dual-branch framework. Features are first extracted by the CLIP image encoder, then transformed into the frequency domain, and finally decoupled into high- and low-frequency paths for complementary modeling of structure and semantics. The high-frequency branch is equipped with a fine-grained structural attention module (FSAM) to enhance textures and edges for detecting small anomalies, while the low-frequency branch uses a global structural context module (GSCM) to capture long-range dependencies and preserve semantic consistency. Together, these branches balance fine detail and global semantics. HarmoniAD further adopts a multi-class joint training strategy, and experiments on MVTec-AD, VisA, and BTAD show state-of-the-art performance with both sensitivity and robustness.

Figures (8)

• Figure 1: Examples of failure cases in existing anomaly detection methods. Yellow boxes denote true anomalies, and red boxes indicate false positives.
• Figure 2: Overall framework of HarmoniAD. CLIP image embeddings are first transformed into the frequency domain and split by a soft gate into high- and low-frequency streams. These streams are reconstructed by the fine-grained structural attention module (FSAM) and the global structural context module (GSCM), respectively, and then perceived as the final representation. The perceived representation is contrasted with the original embeddings to derive patch-level anomaly scores.
• Figure 3: With multi-class joint training, anomaly localization results are presented on selected categories from the MVTec-AD, VisA, and BTAD datasets. Each column is a test sample. The first row shows ground-truth defects (yellow boxes). Rows 3-8 are heatmaps from comparison methods. Rows 9-10 show our patch-level and pixel-level heatmaps. Our method yields high responses in anomaly regions and low responses elsewhere, and outperforms comparison methods on challenging cases.
• Figure 4: Ablation study of F2S Attn. Red boxes denote true anomalies.
• Figure 5: Sensitivity analysis of core loss weights on BTAD.