A Unified Attention U-Net Framework for Cross-Modality Tumor Segmentation in MRI and CT

TL;DR

This study presents a unified Attention U-Net architecture trained jointly on MRI (BraTS 2021) and CT (LIDC-IDRI) datasets to investigate the generalizability of a single model across diverse imaging modalities and anatomical sites, thereby establishing a robust and reproducible baseline for future research in cross-modality tumor segmentation.

Abstract

This study presents a unified Attention U-Net architecture trained jointly on MRI (BraTS 2021) and CT (LIDC-IDRI) datasets to investigate the generalizability of a single model across diverse imaging modalities and anatomical sites. Our proposed pipeline incorporates modality-harmonized preprocessing, attention-gated skip connections, and a modality-aware Focal Tversky loss function. To the best of our knowledge, this study is among the first to evaluate a single Attention U-Net trained simultaneously on separate MRI (BraTS) and CT (LIDC-IDRI) tumor datasets, without relying on modality-specific encoders or domain adaptation. The unified model demonstrates competitive performance in terms of Dice coefficient, IoU, and AUC on both domains, thereby establishing a robust and reproducible baseline for future research in cross-modality tumor segmentation.

Figures (5)

• Figure 1: Representative slices from the datasets utilized in this study a. BraTS 2021 MRI slice (FLAIR modality) with an overlaid tumor mask. b. LIDC-IDRI CT slice displaying the lung nodule segmentation mask.
• Figure 2: Architectural schematic of the proposed Attention U-Net model. The Input/Output blocks (gray) define the data ingestion and final segmentation layers. The Encoder (orange) functions as the contracting path, compressing spatial dimensions to extract hierarchical features. The Bottleneck (red) captures high-level semantic representations, incorporating dropout for regularization. The Decoder (blue) serves as an expanding path, upsampling feature maps to reconstruct the segmentation mask. Attention Gates (green) selectively filter encoder feature maps prior to concatenation to suppress irrelevant activations, while Skip Connections (dashed lines) preserve critical spatial details.
• Figure 3: Training and validation curves showing a. loss and b. Dice score across epochs for both MRI (BraTS) and CT (LIDC-IDRI).
• Figure 4: Performance analysis of the unified Attention U-Net framework on both datasets. (a) Unified ROC curve for test set (AUC = 0.89). (b) Unified confusion matrix.
• Figure 5: Qualitative segmentation results for the unified model. The top panel a. displays results for BraTS Sample 2, and the bottom panel b. for LIDC Sample 4. Each sample row presents the input image, ground truth mask, and predicted mask to visualize segmentation performance on MRI and CT modalities.