Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Brain Tumor Segmentation in MRI Images with 3D U-Net and Contextual Transformer

Thien-Qua T. Nguyen, Hieu-Nghia Nguyen, Thanh-Hieu Bui, Thien B. Nguyen-Tat, Vuong M. Ngo

TL;DR

This work addresses accurate brain tumor segmentation in multi-modal MRI by integrating a 3D Contextual Transformer (CoT) into a 3D-UNet backbone to capture long-range dependencies across modalities. The 3D CoT module computes static and dynamic contextual keys to enhance self-attention, while a Dice plus cross-entropy loss jointly optimizes segmentation. On BraTS2019, the proposed 3D-UNet+CoT achieves ET $82.0\%$, TC $81.5\%$, and WT $89.0\%$ Dice scores, with a mean Dice improvement of $2.2\%$ and $HD_{95}$ reduction of $1.1$ mm, outperforming several state-of-the-art methods. The study also analyzes modality contributions and confirms that each MRI modality adds distinct information for accurate delineation, particularly emphasizing the role of T1c and FLAIR. Overall, the approach demonstrates strong accuracy and suggests future work in preprocessing optimization, computational efficiency, and broader medical-imaging applications.

Abstract

This research presents an enhanced approach for precise segmentation of brain tumor masses in magnetic resonance imaging (MRI) using an advanced 3D-UNet model combined with a Context Transformer (CoT). By architectural expansion CoT, the proposed model extends its architecture to a 3D format, integrates it smoothly with the base model to utilize the complex contextual information found in MRI scans, emphasizing how elements rely on each other across an extended spatial range. The proposed model synchronizes tumor mass characteristics from CoT, mutually reinforcing feature extraction, facilitating the precise capture of detailed tumor mass structures, including location, size, and boundaries. Several experimental results present the outstanding segmentation performance of the proposed method in comparison to current state-of-the-art approaches, achieving Dice score of 82.0%, 81.5%, 89.0% for Enhancing Tumor, Tumor Core and Whole Tumor, respectively, on BraTS2019.

Brain Tumor Segmentation in MRI Images with 3D U-Net and Contextual Transformer

TL;DR

This work addresses accurate brain tumor segmentation in multi-modal MRI by integrating a 3D Contextual Transformer (CoT) into a 3D-UNet backbone to capture long-range dependencies across modalities. The 3D CoT module computes static and dynamic contextual keys to enhance self-attention, while a Dice plus cross-entropy loss jointly optimizes segmentation. On BraTS2019, the proposed 3D-UNet+CoT achieves ET , TC , and WT Dice scores, with a mean Dice improvement of and reduction of mm, outperforming several state-of-the-art methods. The study also analyzes modality contributions and confirms that each MRI modality adds distinct information for accurate delineation, particularly emphasizing the role of T1c and FLAIR. Overall, the approach demonstrates strong accuracy and suggests future work in preprocessing optimization, computational efficiency, and broader medical-imaging applications.

Abstract

This research presents an enhanced approach for precise segmentation of brain tumor masses in magnetic resonance imaging (MRI) using an advanced 3D-UNet model combined with a Context Transformer (CoT). By architectural expansion CoT, the proposed model extends its architecture to a 3D format, integrates it smoothly with the base model to utilize the complex contextual information found in MRI scans, emphasizing how elements rely on each other across an extended spatial range. The proposed model synchronizes tumor mass characteristics from CoT, mutually reinforcing feature extraction, facilitating the precise capture of detailed tumor mass structures, including location, size, and boundaries. Several experimental results present the outstanding segmentation performance of the proposed method in comparison to current state-of-the-art approaches, achieving Dice score of 82.0%, 81.5%, 89.0% for Enhancing Tumor, Tumor Core and Whole Tumor, respectively, on BraTS2019.
Paper Structure (14 sections, 7 equations, 7 figures, 1 table)

This paper contains 14 sections, 7 equations, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related work
  3. Method
  4. 3D Contextual Transformer (CoT)
  5. 3D-UNet model and Loss function
  6. Dataset and Experiments
  7. Evaluation Metrics
  8. Implementation details
  9. Results and Discussion
  10. Ablation study
  11. Evaluation of the influence of each modality
  12. Comparison with state-of-the-arts
  13. Error analysis
  14. Conclusion and future work

Figures (7)

  • Figure 1: This figure displays modalities in two distinct cases, illustrating how these various modalities distinctly delineate different regions of tumor. The blue, red, yellow denote tumor core, enhancing tumor and peritumoral edema, respectively
  • Figure 2: The architectural framework outlines our proposed approach for segmenting brain tumors from MRI images, utilizing the 3D-UNet architecture. (a). Represents the 3D-UNet backbone model. (b). Depicts the 3D contextual transformer (CoT) block directly linked to the convolutional layer
  • Figure 3: The differences between various components are visually compared, showcasing their effectiveness through good cases on the validation set BraTS2019. The variations are represented by dash-squares. The yellow, red, green regions denote the tumor core, the enhancing tumors and peritumoral edema, respectively
  • Figure 4: Performances comparison with some SOTA on the validation set BraTS2019. All metrics are provided by the author
  • Figure 5: Comparison of parameter count and training time for each model (training time per epoch)
  • ...and 2 more figures