Generative Adversarial Networks for Weakly Supervised Generation and Evaluation of Brain Tumor Segmentations on MR Images

TL;DR

This work addresses the high annotation burden in brain tumor segmentation by proposing a weakly supervised framework that uses binary image-level labels to guide 2D MR image segmentation. It combines localization seeds from a classifier with GAN-generated non-cancerous variants to produce tumor segmentations, and introduces a weakly supervised confidence measure to filter segmentations for downstream radiomics-based classification. On BraTS 2020, the method achieves Dice coefficients around 83.9–86.7% and an AUC of 93.32% for radiomics-based pathology classification, closely approaching the performance obtained with true segmentations (AUC 95.80%). The approach reduces radiologist workload and demonstrates practical utility for classification tasks, with potential extensions to 3D and other imaging modalities.

Abstract

Segmentation of regions of interest (ROIs) for identifying abnormalities is a leading problem in medical imaging. Using machine learning for this problem generally requires manually annotated ground-truth segmentations, demanding extensive time and resources from radiologists. This work presents a weakly supervised approach that utilizes binary image-level labels, which are much simpler to acquire, to effectively segment anomalies in 2D magnetic resonance images without ground truth annotations. We train a generative adversarial network (GAN) that converts cancerous images to healthy variants, which are used along with localization seeds as priors to generate improved weakly supervised segmentations. The non-cancerous variants can also be used to evaluate the segmentations in a weakly supervised fashion, which allows for the most effective segmentations to be identified and then applied to downstream clinical classification tasks. On the Multimodal Brain Tumor Segmentation (BraTS) 2020 dataset, our proposed method generates and identifies segmentations that achieve test Dice coefficients of 83.91%. Using these segmentations for pathology classification results with a test AUC of 93.32% which is comparable to the test AUC of 95.80% achieved when using true segmentations.

Figures (8)

• Figure 1: Block diagram visualizing the overall method. Solid gray lines indicate direct inputs and outputs. Dashed gray lines indicate use in loss functions.
• Figure 2: MR image FLAIR channel (left) and its corresponding positive seeds in green, negative seeds in magenta, and unseeded regions in black (right).
• Figure 3: Architectures for generator $G()$, encoder $E()$, and converting U-Net model $U()$, as well as the calculation of the reconstruction loss $L_{recon}$.
• Figure 4: System for training the segmentation model using the seed loss, variation reconstruction loss, and size loss.
• Figure 5: Examples of MR image FLAIR channel (left) and their generated non-cancerous variant (right).