Unsupervised Tumor-Aware Distillation for Multi-Modal Brain Image Translation

TL;DR

The paper addresses the challenge of translating multi-modal brain MRI images when modalities are missing and tumor geometry must be preserved. It introduces UTAD-Net, a two-branch teacher-student distillation framework that uses tumor masks to guide tumor-area translation in an unsupervised setting, with a local consistency loss to preserve anatomy. The teacher processes masked tumor regions to learn tumor-focused representations, which are distilled into a mask-free student capable of producing high-quality target-modality images. On BRATS2020, UTAD-Net achieves competitive quantitative and qualitative results compared with state-of-the-art baselines and demonstrates improved downstream segmentation performance, indicating practical clinical value.

Abstract

Multi-modal brain images from MRI scans are widely used in clinical diagnosis to provide complementary information from different modalities. However, obtaining fully paired multi-modal images in practice is challenging due to various factors, such as time, cost, and artifacts, resulting in modality-missing brain images. To address this problem, unsupervised multi-modal brain image translation has been extensively studied. Existing methods suffer from the problem of brain tumor deformation during translation, as they fail to focus on the tumor areas when translating the whole images. In this paper, we propose an unsupervised tumor-aware distillation teacher-student network called UTAD-Net, which is capable of perceiving and translating tumor areas precisely. Specifically, our model consists of two parts: a teacher network and a student network. The teacher network learns an end-to-end mapping from source to target modality using unpaired images and corresponding tumor masks first. Then, the translation knowledge is distilled into the student network, enabling it to generate more realistic tumor areas and whole images without masks. Experiments show that our model achieves competitive performance on both quantitative and qualitative evaluations of image quality compared with state-of-the-art methods. Furthermore, we demonstrate the effectiveness of the generated images on downstream segmentation tasks. Our code is available at https://github.com/scut-HC/UTAD-Net.

Figures (5)

• Figure 1: Examples of brain tumor MRI images from different modalities.
• Figure 2: Overview of the proposed UTAD-Net. a) The generator $G$ comprises two encoder-decoder pairs and a fusion block $F$ to generate the whole image and tumor image. The student network learns the knowledge of the teacher network through distillation learning at both the feature and image levels. b) The generator $G$ tries to reconstruct the whole image and tumor image. The global discriminator $D^g$ determines whether the whole image is real or fake and classifies its modality, while the local discriminator $D^l$ models the tumor image.
• Figure 3: Qualitative evaluations of our model and the other baselines on the BRATS2020 dataset. The four samples are denoted as: a)T1ce$\rightarrow$T1. b)Flair$\rightarrow$T2. c)T1$\rightarrow$T1ce. d)Flair$\rightarrow$T1ce. For each sample, the first row represents the translation of the whole images, the second row depicts the magnified image of the tumor areas, and the third row illustrates the error map between the generated images and the ground truth(GT) images. Red boxes demonstrate that our method generates more realistic images with clearer textures and richer structural details.
• Figure 4: Four architecture schemes of the student network.a) Complete global branch and local branch (UTAD-Net). b) Only global branch. c) Without local tumor encoder. d) Without local tumor decoder.
• Figure 5: The feature error map of the teacher network and the student network. Each pixel represents a channel.