Generative Style Transfer for MRI Image Segmentation: A Case of Glioma Segmentation in Sub-Saharan Africa

TL;DR

This study addresses brain tumor segmentation in Sub-Saharan Africa where MRI quality varies and data are scarce. It compares baseline Optimized U-Net and nnU-Net configurations (2D and 3D full-resolution) on GLI and SSA data, finding no strong domain shift and demonstrating strong cross-validation performance (up to 0.93–0.95 pseudo-Dice) for both 2D and 3D nnU-Net at 300 epochs. It introduces neural style transfer augmentation to adapt high-quality BraTS data to SSA images and finetunes a pretrained 2D full-res nnU-Net on SSA data, achieving improved SSA validation results. The results suggest NST-based augmentation with targeted SSA finetuning can enhance glioma segmentation in SSA, offering a practical path toward better diagnostic tools in low-resource settings while highlighting the need for larger African datasets and potential ensemble strategies for further gains.

Abstract

In Sub-Saharan Africa (SSA), the utilization of lower-quality Magnetic Resonance Imaging (MRI) technology raises questions about the applicability of machine learning methods for clinical tasks. This study aims to provide a robust deep learning-based brain tumor segmentation (BraTS) method tailored for the SSA population using a threefold approach. Firstly, the impact of domain shift from the SSA training data on model efficacy was examined, revealing no significant effect. Secondly, a comparative analysis of 3D and 2D full-resolution models using the nnU-Net framework indicates similar performance of both the models trained for 300 epochs achieving a five-fold cross-validation score of 0.93. Lastly, addressing the performance gap observed in SSA validation as opposed to the relatively larger BraTS glioma (GLI) validation set, two strategies are proposed: fine-tuning SSA cases using the GLI+SSA best-pretrained 2D fullres model at 300 epochs, and introducing a novel neural style transfer-based data augmentation technique for the SSA cases. This investigation underscores the potential of enhancing brain tumor prediction within SSA's unique healthcare landscape.

Figures (5)

• Figure 1: Fine tuning small-scale SSA data with neural style transfer data augmentation techniques (B) with pretrained weights from large-scale GLI data via transfer learning (A).
• Figure 1: Excluded SSA training Cases A) 00051, B) 00097, C) 00041, & D) 00084 only for the optimized UNet baseline experiment [Predicted masks (top row), T2 (second row), T1ce (third row),T1 (fourth row), and T2-FLAIR (bottom row)]. This was not employed for the rest of the experiments. The predicted masks are shown with color coding as follows: background: purple, necrotic tumor core (NCR): blue, enhancing tumor (ET): yellow, peritumoral edematous tissue (ED): turquoise.
• Figure 2: Predicted Masks for validation data BraTS-GLI-00001-000 (A), BraTS-GLI-00001-001 (B), BraTS-GLI-00013-000 (C), and BraTS-GLI-00013-001 (D), [T1 (top row), T2 (bottom row)] cases using the well-performing best 2D fullres nnUNet model without fine-tuning. The predicted masks (bottom row) are shown with color coding as follows: background: purple, necrotic tumor core (NCR): blue, enhancing tumor (ET): yellow, peritumoral edematous tissue (ED): turquoise.
• Figure 3: Examples of Neural style transfer results between the high-resolution GLI MRI images (style image) and the low-resolution SSA MRI images (content image) via one-to-one random pairing. This was used a data augmentation approach.
• Figure 4: Tumor segmentation improvement for SSA validation case BraTS-SSA-00192-000 before (A) and after (B) neural style transfer data augmentation. [T1 (bottom row), T2(middle row)]. This was also after fine-tuning on SSA training data only from the best GLI pretrained model at the best 2D fullres nn-Unet. The color coding is the following: background: purple, necrotic tumor core (NCR): blue, enhancing tumor (ET): yellow, peritumoral edematous tissue (ED): turquoise.