U2AD: Uncertainty-based Unsupervised Anomaly Detection Framework for Detecting T2 Hyperintensity in MRI Spinal Cord

TL;DR

This work tackles automatic detection of T2-weighted spinal cord hyperintensities with a novel uncertainty-guided unsupervised anomaly detection framework, $U^{2}AD$. By leveraging a Vision Transformer architecture in a mask-and-reconstruct paradigm and integrating both epistemic and aleatoric uncertainty via Monte Carlo sampling, the method adapts within the same clinical dataset to counter domain shifts and task conflicts. A two-stage process—pretraining on healthy data followed by uncertainty-guided adaptation—enables robust reconstruction of normal regions while amplifying anomalies, yielding superior patient-level and segment-level performance compared to both supervised and other unsupervised methods. The approach demonstrates clinical utility for detecting and localizing T2 hyperintensities and sets a new benchmark for uncertainty-guided UAD in medical imaging, with open-source code available for replication and extension.

Abstract

T2 hyperintensities in spinal cord MR images are crucial biomarkers for conditions such as degenerative cervical myelopathy. However, current clinical diagnoses primarily rely on manual evaluation. Deep learning methods have shown promise in lesion detection, but most supervised approaches are heavily dependent on large, annotated datasets. Unsupervised anomaly detection (UAD) offers a compelling alternative by eliminating the need for abnormal data annotations. However, existing UAD methods rely on curated normal datasets and their performance frequently deteriorates when applied to clinical datasets due to domain shifts. We propose an Uncertainty-based Unsupervised Anomaly Detection framework, termed U2AD, to address these limitations. Unlike traditional methods, U2AD is designed to be trained and tested within the same clinical dataset, following a "mask-and-reconstruction" paradigm built on a Vision Transformer-based architecture. We introduce an uncertainty-guided masking strategy to resolve task conflicts between normal reconstruction and anomaly detection to achieve an optimal balance. Specifically, we employ a Monte-Carlo sampling technique to estimate reconstruction uncertainty mappings during training. By iteratively optimizing reconstruction training under the guidance of both epistemic and aleatoric uncertainty, U2AD reduces overall reconstruction variance while emphasizing regions. Experimental results demonstrate that U2AD outperforms existing supervised and unsupervised methods in patient-level identification and segment-level localization tasks. This framework establishes a new benchmark for incorporating uncertainty guidance into UAD, highlighting its clinical utility in addressing domain shifts and task conflicts in medical image anomaly detection. Our code is available: https://github.com/zhibaishouheilab/U2AD

Figures (14)

• Figure 1: Illustration of T2 hyperintensities in spinal cord MR images, the presence of domain shifts between healthy and clinical datasets, and an overview of different methods used for detecting T2 hyperintensities. The top-left panel highlights anomalous examples with T2 hyperintensities, marked by red outlines. The bottom-left panel depicts the domain shift in signal intensity distribution between healthy datasets and clinical datasets, emphasizing the challenges posed by demographic and pathological differences. The right panel outlines four approaches for T2 hyperintensity detection: (1) segmentation models that require precise anomaly annotations, (2) signal analysis methods prone to noise interference, (3) object detection models reliant on large labeled datasets, and (4) reconstruction-based anomaly detection models, which are unsupervised but can struggle with domain shifts.
• Figure 2: Workflow of the proposed $U^{2}AD$ framework. The model is pretrained on a large-scale healthy dataset, followed by a two-stage adaptation on clinical datasets. The Monte Carlo-based uncertainty estimation is integrated into the mask-and-reconstruct process. Epistemic uncertainty and aleatoric uncertainty guides masking and reconstruction training in the adaptation training. The outputs include patient-level anomaly identification and segment-level localization, with quantitative measures of anomaly scores to assess performance.
• Figure 3: Comparison of three masking strategies in the $U^{2}AD$ framework: random masking, epistemic uncertainty (EU)-guided masking, and aleatoric uncertainty (AU)-guided masking. (a) The top-left panel illustrates the random masking strategy, where patches are randomly masked before being input into the ViT encoder-decoder for reconstruction. The middle panel demonstrates the proposed EU-guided masking strategy, leveraging variance maps from MC inference to generate masking probability for each patch, prioritizing low-confidence regions during training. The bottom panel presents the AU-guided masking strategy, where high-error patches in the error map (representing potential anomalies) are excluded from training, thus amplifying reconstruction errors in anomaly regions. (b) Masked patches from all three strategies are processed through the ViT encoder-decoder, resulting in reconstructed images that showcase differences in anomaly detection performance. (c) Highlights the roles of EU- and AU-guided masking in mitigating underfitting and overfitting, achieving balanced reconstruction for normal regions and effective anomaly detection.
• Figure 4: Datasets and tasks description for evaluating the $U^{2}AD$ framework. The Spine Generic T2w dataset (#1) is used for pretraining the MAE model ($\theta_{\text{pre}}$). Adaptation training is applied on the OpenNeuro T2w dataset (#2), RenjiDCM T2w dataset (#3), and RuijinDCM T2w dataset (#4). The OpenNeuro T2w dataset includes pseudo-anomalies and is designed for reconstruction assessment in anomaly-specific regions. The RenjiDCM T2w dataset is used to evaluate reconstruction performance across the entire ROI, while both RenjiDCM T2w (#3) and RuijinDCM T2w (#4) are evaluated for anomaly detection and localization tasks.
• Figure 5: Examples of reconstruction results and anomaly maps generated by different UAD models. For each image, the reconstruction is limited within the ROI mask (the spinal cord). For each example, we compare the reconstruction results and anomaly maps between these UAD models. The true anomaly regions are highlighted with yellow boxes and red arrows, and magnified to better illustrate the results. Also, we present the varied anomaly maps during the training process for DAE, VAE, and $U^{2}AD$. The numerical values above each anomaly map represent the reconstruction error for the entire ROI.