MRI Scan Synthesis Methods based on Clustering and Pix2Pix

TL;DR

This work addresses the practical problem of missing MRI modalities by synthesizing a T2-weighted image from a T1-weighted scan and evaluating its impact on tumor segmentation. It introduces two approaches: BrainClustering, a clustering-based tissue-wise intensity-mapping method, and Pix2Pix, a conditional GAN-based image-to-image translator with medical-imaging adaptations. The results show that synthesized T2W images can closely resemble real scans and support effective segmentation, with Pix2Pix often delivering the best MSE and Dice metrics while BrainClustering remains competitive in some settings; both outperform simple baselines. The findings suggest that modality substitution can meaningfully aid clinical workflows when certain MRI sequences are unavailable, enabling more robust automated segmentation in real-world practice.

Abstract

We consider a missing data problem in the context of automatic segmentation methods for Magnetic Resonance Imaging (MRI) brain scans. Usually, automated MRI scan segmentation is based on multiple scans (e.g., T1-weighted, T2-weighted, T1CE, FLAIR). However, quite often a scan is blurry, missing or otherwise unusable. We investigate the question whether a missing scan can be synthesized. We exemplify that this is in principle possible by synthesizing a T2-weighted scan from a given T1-weighted scan. Our first aim is to compute a picture that resembles the missing scan closely, measured by average mean squared error (MSE). We develop/use several methods for this, including a random baseline approach, a clustering-based method and pixel-to-pixel translation method by Isola et al. (Pix2Pix) which is based on conditional GANs. The lowest MSE is achieved by our clustering-based method. Our second aim is to compare the methods with respect to the effect that using the synthesized scan has on the segmentation process. For this, we use a DeepMedic model trained with the four input scan modalities named above. We replace the T2-weighted scan by the synthesized picture and evaluate the segmentations with respect to the tumor identification, using Dice scores as numerical evaluation. The evaluation shows that the segmentation works well with synthesized scans (in particular, with Pix2Pix methods) in many cases.

Figures (7)

• Figure 1: A slice of a T1W and a T2W scan, and of the T1W complement (Patient BraTS19 CBICA BLI 1).
• Figure 2: [Best viewed in color] Schematic representation of the BrainClustering process: First, we use $k$-means1d to obtain a macro clustering of the scan according to the desired number of macro clusters. Second, each macro cluster is clustered into micro clusters. Finally, the points belonging to these micro clusters are averaged and stored in one table for each tissue, where there is an entry for each corresponding value of T1W and T2W.
• Figure 3: The segmentations in the second and in the fourth row have been created using the respective T2W and the three original T1W, T1CE, FLAIR. The rightmost image in the "Real T1W" column shows the real T1W image and the ground truth segmentation.
• Figure 4: BrainClustering, Pix2Pix and original T2W scans and corresponding superimposed segmentation generated with DeepMedic (together with the real T1W, T1CE and FLAIR), shown for all cases from the testOUR data set. In the last column the ground truth segmentation is superimposed on the T1W scan.
• Figure 5: Continuation of \ref{['fig:testuk1']}.