Spatial-aware Attention Generative Adversarial Network for Semi-supervised Anomaly Detection in Medical Image

TL;DR

This work introduces SAGAN, a Spatial-aware Attention Generative Adversarial Network for semi-supervised anomaly detection in medical images. By integrating patch-based positional encoding and an attention-driven restoration mechanism, SAGAN restores normal structures while preserving anatomical consistency, using unlabeled data to generate both healthy and pseudo-anomalous mappings. The model computes anomaly scores from the discrepancy between input images and their generated healthy counterparts, achieving state-of-the-art results on RSNA, VinDr-CXR, and LAG datasets and showing robustness to varying anomaly ratios. These advances have practical impact for efficiently flagging anomalies in medical imaging without requiring extensive labeled anomaly data, though pixel-level localization remains a challenge and further work on annotation-free localization is planned.

Abstract

Medical anomaly detection is a critical research area aimed at recognizing abnormal images to aid in diagnosis.Most existing methods adopt synthetic anomalies and image restoration on normal samples to detect anomaly. The unlabeled data consisting of both normal and abnormal data is not well explored. We introduce a novel Spatial-aware Attention Generative Adversarial Network (SAGAN) for one-class semi-supervised generation of health images.Our core insight is the utilization of position encoding and attention to accurately focus on restoring abnormal regions and preserving normal regions. To fully utilize the unlabelled data, SAGAN relaxes the cyclic consistency requirement of the existing unpaired image-to-image conversion methods, and generates high-quality health images corresponding to unlabeled data, guided by the reconstruction of normal images and restoration of pseudo-anomaly images.Subsequently, the discrepancy between the generated healthy image and the original image is utilized as an anomaly score.Extensive experiments on three medical datasets demonstrate that the proposed SAGAN outperforms the state-of-the-art methods.

Figures (3)

• Figure 1: Overview of SAGAN. During the training stage, SAGAN learns to retore unlabeled images as normal images with the supervision of reconstructing normal images and restoring pseudo abnormal images. At the testing stage, the difference between the generated image and the original image reveals the presence of anomaly data.
• Figure 2: Generator of our proposed SAGAN. The main idea of the generator is to take advantage of the anatomical consistency of medical images and assign higher weights to the attention of anomaly features to refine the restoration of anomaly regions. Generator utilizes positional conditional encoding of patches to extract positional information and uses attention gate to adaptively learn the attention map of regions of interest to accurately control the restoration of anomaly regions.
• Figure 3: Visualization of heatmaps on medical datasets. The heatmap is derived from the difference maps between the restored image and the original image. Images from top to bottom are from RSNA, VinDr-CXR and LAG datasets, respectively. Original images, restored images, and heatmap visualizations are arranged from left to right. Green boxes represent abnormal regions and red boxes indicate the corresponding abnormal regions on heatmaps.