AnomalyHybrid: A Domain-agnostic Generative Framework for General Anomaly Detection

TL;DR

AnomalyHybrid introduces a domain-agnostic, GAN-based framework that generates authentic and diverse anomalies by conditioning on color, depth, and edge cues using two decoders. It operates in an unsupervised manner on sets of RGB, depth, and edge maps extracted from the same images, enabling both RGB-level and depth-level anomaly generation across industrial and biological domains. The method achieves state-of-the-art generation metrics (IS and LPIPS) and consistently improves downstream anomaly classification, detection, and segmentation on datasets such as MVTecAD, MVTec3D, and Heliconius Butterfly. Its design, relying on depth- and edge-conditioned decoders and simple augmentations, offers strong generalization and potential applicability to related tasks beyond anomaly synthesis.

Abstract

Anomaly generation is an effective way to mitigate data scarcity for anomaly detection task. Most existing works shine at industrial anomaly generation with multiple specialists or large generative models, rarely generalizing to anomalies in other applications. In this paper, we present AnomalyHybrid, a domain-agnostic framework designed to generate authentic and diverse anomalies simply by combining the reference and target images. AnomalyHybrid is a Generative Adversarial Network(GAN)-based framework having two decoders that integrate the appearance of reference image into the depth and edge structures of target image respectively. With the help of depth decoders, AnomalyHybrid achieves authentic generation especially for the anomalies with depth values changing, such a s protrusion and dent. More, it relaxes the fine granularity structural control of the edge decoder and brings more diversity. Without using annotations, AnomalyHybrid is easily trained with sets of color, depth and edge of same images having different augmentations. Extensive experiments carried on HeliconiusButterfly, MVTecAD and MVTec3D datasets demonstrate that AnomalyHybrid surpasses the GAN-based state-of-the-art on anomaly generation and its downstream anomaly classification, detection and segmentation tasks. On MVTecAD dataset, AnomalyHybrid achieves 2.06/0.32 IS/LPIPS for anomaly generation, 52.6 Acc for anomaly classification with ResNet34, 97.3/72.9 AP for image/pixel-level anomaly detection with a simple UNet.

Figures (13)

• Figure 1: AnomalyHybrid is a domain-agnostic generative framework. Unlike prior industrial anomaly specialists, it generates general anomalies simply by combining the reference(green arrows) and target(yellow arrows) images.
• Figure 2: Inference workflow of AnomalyHybrid. AnomalyHybrid combines the appearance of reference image with the depth and edge structural target image. It generates global and local anomalies without and with applying manipulations on target depth and edge maps.
• Figure 3: Comparison of related frameworks. (a) summarizes the three key components in model-free anomaly synthesis methods, such as Draem draem and 3DSR 3dsr. (b) relies on large diffusion model. (c) achieves authentic anomaly generation by learning multiple defect-aware specialists. Comparing to previous workflows, (d) our proposed AnomalyHybrid has more comprehensive generation ability.
• Figure 4: Training workflow of AnomalyHybrid. AnomalyHybrid is trained with sets of depth, color, edge of same images but having different augmentations. It consists of an encoder, two decoders and a discriminator. All decoders consist of anomaly texture and mask branches. The two-branch architecture forces the network to inject the appearance of reference to the structural of target depth and edge.
• Figure 5: Examples of anomaly generation using different manipulations on Hazelnut of MVTecAD mvtec.