Detecting Outliers with Foreign Patch Interpolation

TL;DR

This work tackles the challenge of detecting subtle medical outliers by leveraging self-supervised learning. It introduces Foreign Patch Interpolation (FPI), where a patch from one image is convexly combined with a patch from another to create synthetic irregularities, and a wide residual encoder-decoder is trained to predict the patch location and the interpolation factor $\alpha$ for pixel-level localization and an anomaly score. The method ranks highly on the 2020 MICCAI MOOD challenge and demonstrates strong performance on the DeepLesion dataset, outperforming several unsupervised baselines while remaining robust to varied anatomy and alignment. The results suggest that focusing on local, interpolated foreign patterns enables effective detection of subtle abnormalities without requiring abnormal training data, offering a practical tool to assist radiologists with automated screening and triage.

Abstract

In medical imaging, outliers can contain hypo/hyper-intensities, minor deformations, or completely altered anatomy. To detect these irregularities it is helpful to learn the features present in both normal and abnormal images. However this is difficult because of the wide range of possible abnormalities and also the number of ways that normal anatomy can vary naturally. As such, we leverage the natural variations in normal anatomy to create a range of synthetic abnormalities. Specifically, the same patch region is extracted from two independent samples and replaced with an interpolation between both patches. The interpolation factor, patch size, and patch location are randomly sampled from uniform distributions. A wide residual encoder decoder is trained to give a pixel-wise prediction of the patch and its interpolation factor. This encourages the network to learn what features to expect normally and to identify where foreign patterns have been introduced. The estimate of the interpolation factor lends itself nicely to the derivation of an outlier score. Meanwhile the pixel-wise output allows for pixel- and subject- level predictions using the same model.

Figures (12)

• Figure 1: Examples of foreign patch interpolation in brain and abdominal data from the MOOD challenge (zimmerer2020medical). Different $\alpha$ values correspond to different convex combinations. Scaling the label mask by the $\alpha$ value gives the label for each example. Green markers indicate the corners of the patch. Regions where $A$ and $A'$ are equal, e.g., background, are truncated in the label according to Eqn. \ref{['equation:alpha_label']}. More examples are given in Appendix \ref{['appendix:fpiBrain']} and \ref{['appendix:fpiAbdomen']}.
• Figure 2: Example of sink/source deformation used to synthesize an outlier. Original sample from MOOD challenge (zimmerer2020medical). All types of synthetic outliers are displayed in Appendix \ref{['appendix:synthetic']}.
• Figure 3: Image-level abnormality scores for slices throughout the volume. Showing sink/source outlier (red plot and top images) and normal sample (blue line and bottom images). Data from MOOD (zimmerer2020medical). Slices with deformation have high anomaly scores (red), concentrated around the bulbous deformation (top left image). Normal sample has minimal abnormality scores.
• Figure 4: Average precision for MOOD brain data (zimmerer2020medical) using different model configurations. The binary and continuous round-up models serve as simplified methods used in our ablation study. The continuous and discrete models represent our standard method. The addition of SWA is an optional extension.
• Figure 5: Examples of global outliers using MOOD data (zimmerer2020medical). (a) Original normal sample (top left) and rotations. (b) Gaussian blur ($\sigma=1$) and abdominal data. Note the change of scale in activation maps. Plots display the abnormality score across slices.