Learning Global and Local Features of Normal Brain Anatomy for Unsupervised Abnormality Detection

TL;DR

This study demonstrates an unsupervised learning framework for pixel-wise abnormality detection in brain magnetic resonance imaging captured from a patient population with metastatic brain tumor and devise a metric to evaluate the anatomical legitimacy of the reconstructed images and shows that the overall detection performance is improved when the image reconstruction network achieves a higher score.

Abstract

In real-world clinical practice, overlooking unanticipated findings can result in serious consequences. However, supervised learning, which is the foundation for the current success of deep learning, only encourages models to identify abnormalities that are defined in datasets in advance. Therefore, abnormality detection must be implemented in medical images that are not limited to a specific disease category. In this study, we demonstrate an unsupervised learning framework for pixel-wise abnormality detection in brain magnetic resonance imaging captured from a patient population with metastatic brain tumor. Our concept is as follows: If an image reconstruction network can faithfully reproduce the global features of normal anatomy, then the abnormal lesions in unseen images can be identified based on the local difference from those reconstructed as normal by a discriminative network. Both networks are trained on a dataset comprising only normal images without labels. In addition, we devise a metric to evaluate the anatomical fidelity of the reconstructed images and confirm that the overall detection performance is improved when the image reconstruction network achieves a higher score. For evaluation, clinically significant abnormalities are comprehensively segmented. The results show that the area under the receiver operating characteristics curve values for metastatic brain tumors, extracranial metastatic tumors, postoperative cavities, and structural changes are 0.78, 0.61, 0.91, and 0.60, respectively.

Figures (7)

• Figure 1: Schematic illustration of image reconstruction network for learning global features of normal brain anatomy. (a) Architecture comprising an encoder network and a decoder network. The input image $\bm{x}$ is mapped to a low-dimensional latent space through the encoder. The decoder generates its reconstruction $\bm{\hat{x}}$ from the sampled latent variable $\bm{z}$. In the learning framework of IntroVAE, the encoder determines whether the input is from the data distribution or the decoder by changing the destination of its mapping function, whereas the decoder generates more realistic images to deceive the encoder. (b) Latent representation search obtains a better latent representation $\bm{z_*}$ for a better reconstructed image $\bm{\hat{x}_*}$ with lower reconstruction error.
• Figure 2: Discriminative networks for recognizing local patterns of normal brain anatomy. (a) Discriminative networks learn patch-wise discriminative embeddings based on metric learning techniques using triplet margin loss. (b) By calculating the patch-wise similarity in discriminative embeddings between unseen images and reconstructed normal-appearing images, the abnormality distribution can be measured as abnormality scores.
• Figure 3: Dataset splitting. In total, 275 cases were included in the present study. From 235 cases with no history of brain surgery, 200 cases were randomly selected, and the data were separated into 36,075 axial slices. Each slice was independently grouped based on the presence of any abnormality from the four classes. Among those slices, 29,278 slices with no annotated abnormalities were assigned to the training dataset. The remaining 35 patients with no history of brain surgery and 40 patients with a history of surgery were integrated into a test dataset.
• Figure 4: Qualitative and quantitative comparison of image reconstruction results of VAE, IntroVAE, and IntroVAE+LatSearch. (a) Input images, images generated by VAE, IntroVAE, and IntroVAE+LatSearch are shown on the first to the fourth row, respectively. These images confirm that fine details were well reproduced using IntroVAE instead of VAE as the backbone. (b) Pixel-wise softmax entropy provided by the segmentation network is shown in the second row. Higher values appear in obscure regions of the generated images, especially those generated by VAE. As shown in the third row, anatomical consistency can be assessed by calculating the concordance in segmentation labels between the input and reconstructed images.
• Figure 5: Difference between abnormality scores and per-pixel distance of intensities. Note that the per-pixel L1 distance was extremely sensitive for indistinct image differences, whereas the abnormality score generally reflects the presence of a semantic object (see the region of metastatic brain tumor indicated by the red label). In particular, the accumulation of abnormal scores correlated well with the lesion when using IntroVAE+LatSearch.