Efficient Slice Anomaly Detection Network for 3D Brain MRI Volume

TL;DR

This work introduces SimpleSliceNet, a lightweight 2D-slice based approach for unsupervised anomaly detection in 3D brain MRI volumes. By fine-tuning an ImageNet-pretrained slice feature extractor via Encoder-Decoder Contrast and modeling feature likelihood with a Conditional Normalizing Flow under a Semi-Push-Pull regime, the method achieves strong localization and competitive detection performance while substantially reducing memory and computation compared to 3D reconstruction-based models. The framework processes volumes slice-by-slice and aggregates features to form robust 3D representations, enabling accurate anomaly localization with fewer false positives. Experiments on Br35H, IXI, and BraTS2021 demonstrate favorable trade-offs in accuracy, speed, and resource usage, highlighting potential for clinically practical deployment.

Abstract

Current anomaly detection methods excel with benchmark industrial data but struggle with natural images and medical data due to varying definitions of 'normal' and 'abnormal.' This makes accurate identification of deviations in these fields particularly challenging. Especially for 3D brain MRI data, all the state-of-the-art models are reconstruction-based with 3D convolutional neural networks which are memory-intensive, time-consuming and producing noisy outputs that require further post-processing. We propose a framework called Simple Slice-based Network (SimpleSliceNet), which utilizes a model pre-trained on ImageNet and fine-tuned on a separate MRI dataset as a 2D slice feature extractor to reduce computational cost. We aggregate the extracted features to perform anomaly detection tasks on 3D brain MRI volumes. Our model integrates a conditional normalizing flow to calculate log likelihood of features and employs the Semi-Push-Pull Mechanism to enhance anomaly detection accuracy. The results indicate improved performance, showcasing our model's remarkable adaptability and effectiveness when addressing the challenges exists in brain MRI data. In addition, for the large-scale 3D brain volumes, our model SimpleSliceNet outperforms the state-of-the-art 2D and 3D models in terms of accuracy, memory usage and time consumption. Code is available at: https://anonymous.4open.science/r/SimpleSliceNet-8EA3.

Figures (5)

• Figure 1: Samples from datasets This figure displays samples from each dataset used in this study. Each row represents samples from a different dataset, providing a visual comparison of their characteristics and preprocessing differences. Samples from the IXI and BraTS2021 shown in this figure have undergone preprocessing.
• Figure 2: SimpleSliceNet overall architecture Overview of the proposed SimpleSliceNet. The slice encoder $\boldsymbol{E}$ utilizes a backbone pre-trained on ImageNet to extract multi-layer feature maps $\boldsymbol{f_i}$ at low resolution. Anomalies are synthesized by introducing Gaussian noise into the feature space. Subsequently, these features are fed into CNF to estimate the log likelihood of the normal and anomalous distribution, and employ a SPP loss to refine the explicit boundaries of the normal distribution. Our SimpleSliceNet extracts slice features using the slice encoder and aggregates the resulting slice features via permutation invariant operations to achieve the features $\boldsymbol{F^i}$ of the volume $\boldsymbol{v^i}$ in low resolution. We process the feature vector of each voxel exactly like what we do on 2D slices to obtain anomaly map $\boldsymbol{s(v^i)}$. We finally upscale the anomaly map to the same resolution as that of the original volume.
• Figure 3: Qualitative Results on the SOTA methods This figure displays slices from volumes selected from the testing dataset, chosen to represent different levels within the volume and varied visual structures. This selection ensures a more comprehensive evaluation of the models' performance. The first column presents the images (slices) from brain MRI volumes and the second column illustrates the ground-truth data for anomalous regions. The subsequent columns depict the binary anomaly map predicted by selected models. These maps are generated by applying the F1 threshold, which yield the best F1 score, to convert anomaly scores into binary masks.
• Figure 4: Qualitative Results on the SOTA methods This figure displays intermediate slices in sequence from a single volume selected from the testing dataset. The visualization is designed to demonstrate how well the model performs spatially across an entire volume. The first row presents the images (slices) from brain MRI volumes and the second row shows the ground-truth for the anomalous regions. The remaining rows depict the binary anomaly maps predicted by SOTA models. These maps are generated by applying the F1 threshold, which achieves the best F1 score, to convert anomaly scores into binary masks.
• Figure 5: Plots of AUROC and loss during training This figure presents plots of image-level and pixel-level AUROC, along with average training loss, observed during the training phase. For practicality, we trained on 60 volumes from the training dataset and used 20 volumes from the validation dataset for evaluation. We assessed model performance after processing each volume (every single iteration), and the results for the first 250 seconds are plotted. The interval between two consecutive points on a line indicates the time required for the model to process one volume, consisting of 96 slices. A: plot of pixel-level AUROC. B: plot of image-level AUROC. C: plot of average training loss.