Localizing Anomalies via Multiscale Score Matching Analysis

TL;DR

The paper addresses unsupervised anomaly localization in medical imaging, focusing on brain MRI. It introduces Spatial-MSMA, an extension of Multiscale Score Matching Analysis that adds spatial conditioning via patch-wise conditional likelihoods estimated with a normalizing-flow model, producing anomaly heatmaps. On a dataset of 1,650 pediatric MRIs with simulated lesions, Spatial-MSMA outperforms reconstruction-, generative-, and attribution-based baselines across distance metrics (e.g., 99th percentile Hausdorff distance and Mean Surface Distance) and component-wise metrics (TPR, PPV). The approach yields accurate, interpretable localization with clinical relevance, and code is released for public use.

Abstract

Anomaly detection and localization in medical imaging remain critical challenges in healthcare. This paper introduces Spatial-MSMA (Multiscale Score Matching Analysis), a novel unsupervised method for anomaly localization in volumetric brain MRIs. Building upon the MSMA framework, our approach incorporates spatial information and conditional likelihoods to enhance anomaly detection capabilities. We employ a flexible normalizing flow model conditioned on patch positions and global image features to estimate patch-wise anomaly scores. The method is evaluated on a dataset of 1,650 T1- and T2-weighted brain MRIs from typically developing children, with simulated lesions added to the test set. Spatial-MSMA significantly outperforms existing methods, including reconstruction-based, generative-based, and interpretation-based approaches, in lesion detection and segmentation tasks. Our model achieves superior performance in both distance-based metrics (99th percentile Hausdorff Distance: $7.05 \pm 0.61$, Mean Surface Distance: $2.10 \pm 0.43$) and component-wise metrics (True Positive Rate: $0.83 \pm 0.01$, Positive Predictive Value: $0.96 \pm 0.01$). These results demonstrate Spatial-MSMA's potential for accurate and interpretable anomaly localization in medical imaging, with implications for improved diagnosis and treatment planning in clinical settings. Our code is available at~\url{https://github.com/ahsanMah/sade/}.

Figures (2)

• Figure 1: Overview of Spatial-MSMA. A neural score estimator produces score tensors at multiple noise scales. The score tensors are divided into patches and processed by a conditional flow to estimate patch-wise anomaly scores. Global image features are extracted by a convolutional network and combined with positional encodings corresponding to each patch location, resulting in a conditioning vector per patch. The patch score norms and conditioning vectors are fed into a normalizing flow model with conditional coupling blocks. The result is a negative likelihood heatmap that highlights anomalous patches within the image. Spatial-MSMA thus enables precise localization of anomalies based on the patch scores and their spatial context.
• Figure 2: Qualitative comparison of anomaly heatmaps across different methods. The first row shows random axial slices of the volumetric input samples. The lesions are highlighted in magenta. Each column is a slice from random individuals. Note how Spatial-MSMA consistently detects all the lesions in the image, while other methods tend to miss smaller lesions.