Unsupervised Patch-GAN with Targeted Patch Ranking for Fine-Grained Novelty Detection in Medical Imaging

TL;DR

This work tackles the problem of detecting unseen medical abnormalities in scenarios with scarce labeled data, particularly when small lesions are embedded in large normal regions. It introduces an unsupervised Patch-GAN that reconstructs masked normal regions and evaluates anomaly likelihood at the patch level, coupled with a patch-ranking mechanism to emphasize high-discrepancy areas within the global image context. The method combines a mask reconstructor, PatchGAN discriminators, and a rank-based weighting of patch losses, achieving state-of-the-art AUCs on ISIC 2016 and BraTS 2019 (95.79% and 96.05%, respectively). These results demonstrate enhanced sensitivity to subtle local anomalies while preserving global coherence, with potential clinical impact in early and accurate anomaly localization. Future work may explore diffusion-based generative models to further improve fine-grained novelty detection in complex medical imaging tasks.

Abstract

Detecting novel anomalies in medical imaging is challenging due to the limited availability of labeled data for rare abnormalities, which often display high variability and subtlety. This challenge is further compounded when small abnormal regions are embedded within larger normal areas, as whole-image predictions frequently overlook these subtle deviations. To address these issues, we propose an unsupervised Patch-GAN framework designed to detect and localize anomalies by capturing both local detail and global structure. Our framework first reconstructs masked images to learn fine-grained, normal-specific features, allowing for enhanced sensitivity to minor deviations from normality. By dividing these reconstructed images into patches and assessing the authenticity of each patch, our approach identifies anomalies at a more granular level, overcoming the limitations of whole-image evaluation. Additionally, a patch-ranking mechanism prioritizes regions with higher abnormal scores, reinforcing the alignment between local patch discrepancies and the global image context. Experimental results on the ISIC 2016 skin lesion and BraTS 2019 brain tumor datasets validate our framework's effectiveness, achieving AUCs of 95.79% and 96.05%, respectively, and outperforming three state-of-the-art baselines.

Figures (6)

• Figure 1: Overview of the proposed one-class classification framework, illustrating the key components for novelty detection, including patch-level reconstruction and targeted patch ranking.
• Figure 2: Examples of training images used in the model: (top row) original normal images from the brain (left) and skin (right) datasets; (bottom row) corresponding masked images used for self-supervised learning, where black boxes denote masked regions.
• Figure 3: Distribution of abnormal scores for AE, ALOCC, OCSVM, and the proposed method on the BraTS 2019 and ISIC 2016 datasets (left to right), illustrating the separation between normal and abnormal classes across methods.
• Figure 4: ISIC 2016 samples: normal (left) and abnormal (right).
• Figure 5: BraTS 2019 samples: normal (left) and abnormal (right).