Towards Universal Unsupervised Anomaly Detection in Medical Imaging

TL;DR

The paper tackles the problem of universal unsupervised anomaly detection in medical imaging, addressing the bias of supervised methods toward predefined pathologies. It introduces Reversed Auto-Encoders (RA), a generative framework trained on normal anatomy to reconstruct pseudo-healthy inputs ($x_{ph}$) and identify anomalies across multiple modalities by computing robust residual and perceptual differences. RA leverages a multi-scale reversed embedding similarity loss and an introspective encoder–decoder dynamic, along with an adaptive histogram equalization–based anomaly score, to achieve broad anomaly coverage. Extensive evaluation across brain MRI, pediatric wrist X-rays, and chest X-rays demonstrates that RA outperforms several state-of-the-art methods in terms of detection and localization across diverse pathologies, suggesting strong potential for clinical screening and diagnosis; the authors also provide public code for reproducibility and further benchmarking.

Abstract

The increasing complexity of medical imaging data underscores the need for advanced anomaly detection methods to automatically identify diverse pathologies. Current methods face challenges in capturing the broad spectrum of anomalies, often limiting their use to specific lesion types in brain scans. To address this challenge, we introduce a novel unsupervised approach, termed \textit{Reversed Auto-Encoders (RA)}, designed to create realistic pseudo-healthy reconstructions that enable the detection of a wider range of pathologies. We evaluate the proposed method across various imaging modalities, including magnetic resonance imaging (MRI) of the brain, pediatric wrist X-ray, and chest X-ray, and demonstrate superior performance in detecting anomalies compared to existing state-of-the-art methods. Our unsupervised anomaly detection approach may enhance diagnostic accuracy in medical imaging by identifying a broader range of unknown pathologies. Our code is publicly available at: \url{https://github.com/ci-ber/RA}.

Figures (5)

• Figure 1: Towards Universal Unsupervised Anomaly Detection. The figure illustrates the detection of various anomalies in a dataset comprised of $\approx$ 38,000 images, spanning 22 anomaly classes, 3 anatomies, and 2 imaging modalities. The representation underscores the capacity of the model to learn from normal populations and its effectiveness in identifying unknown anomalies highlighting its potential utility in clinical screening and diagnostic applications.
• Figure 2: Reversed Autoencoder (RA) framework during training and inference phases. During the training phase (left), the encoder and decoder networks are optimized using a multi-scale reversed embedding loss $\mathcal{L}_{\text{Reversed}}$, in conjunction with the Evidence Lower Bound (ELBO) and adversarial optimization. In this process, the decoder generates a synthetic image $x_{\text{fake}}$ from random noise, with the goal of fooling the encoder into treating it as a real image. In the inference phase (right), the RA model processes a new input $x$, encoding and reconstructing it into a pseudo-healthy image $x_{\text{ph}}$. Anomaly detection is carried out by computing the L1 norm and perceptual differences between $x$ and $x_{\text{ph}}$, resulting in an anomaly map that highlights pathological regions.
• Figure 3: Anomaly Detection in Brain MRI using Reversed Auto-Encoders (RA). The top row displays original brain MRI scans with expert-annotated pathologies (in red) and additional pathologies (in cyan). The middle row depicts the pseudo-healthy reconstructions generated by RA, while the bottom row presents anomaly maps, with detected pathologies highlighted in brighter colors. The legend on the right details the dataset composition and the range of pathologies evaluated.
• Figure 4: Anomaly detection on pediatric wrist X-rays showcasing a comparison between the original input images, the reconstructed images using our method (RA), and the corresponding anomaly maps. Each column represents a different category of anomaly identified in the study, with expert annotations evaluated, other present pathologies, and areas of interest for zoom-in highlighted. The anomaly maps are color-coded to facilitate the localization and visualization of potential pathologies. The dataset encompasses 17k images and 7 anomaly types, demonstrating the diversity and complexity of the clinical conditions analyzed.
• Figure 5: Anomaly detection on chest X-rays. The figure illustrates a comparison across three panels: normal, pneumonia, and COVID-19 CXRs. For each condition, the top row presents the original input images, the middle row shows the pseudo-healthy reconstructions and the bottom row displays the corresponding anomaly maps. Anomalies are indicated by red boxes in the input images and are highlighted in the anomaly maps to indicate the severity and location of the pathology. The dataset comprises 20k images spanning two anomaly classes.