Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Integrating Edges into U-Net Models with Explainable Activation Maps for Brain Tumor Segmentation using MR Images

Subin Sahayam, Umarani Jayaraman

TL;DR

This work addresses automatic brain tumor segmentation from multi-modal MRIs by proposing to learn tumor edges as additional training targets. Edges are derived from ground truth using a 3D Laplacian like filter and fused with conventional tumor region labels through one hot representations, enabling edge aware supervision across multiple U-Net variants. Across experiments on BraTS2020, edge-target training improves core tumor and edge delineation metrics and yields interpretable edge maps and activation maps, bringing classic architectures closer to state-of-the-art models like Swin U-Net and Hybrid MR-U-Net. The approach offers a simple, data oriented way to enhance segmentation precision and provide clinically useful edge information for planning and intervention.

Abstract

Manual delineation of tumor regions from magnetic resonance (MR) images is time-consuming, requires an expert, and is prone to human error. In recent years, deep learning models have been the go-to approach for the segmentation of brain tumors. U-Net and its' variants for semantic segmentation of medical images have achieved good results in the literature. However, U-Net and its' variants tend to over-segment tumor regions and may not accurately segment the tumor edges. The edges of the tumor are as important as the tumor regions for accurate diagnosis, surgical precision, and treatment planning. In the proposed work, the authors aim to extract edges from the ground truth using a derivative-like filter followed by edge reconstruction to obtain an edge ground truth in addition to the brain tumor ground truth. Utilizing both ground truths, the author studies several U-Net and its' variant architectures with and without tumor edges ground truth as a target along with the tumor ground truth for brain tumor segmentation. The author used the BraTS2020 benchmark dataset to perform the study and the results are tabulated for the dice and Hausdorff95 metrics. The mean and median metrics are calculated for the whole tumor (WT), tumor core (TC), and enhancing tumor (ET) regions. Compared to the baseline U-Net and its variants, the models that learned edges along with the tumor regions performed well in core tumor regions in both training and validation datasets. The improved performance of edge-trained models trained on baseline models like U-Net and V-Net achieved performance similar to baseline state-of-the-art models like Swin U-Net and hybrid MR-U-Net. The edge-target trained models are capable of generating edge maps that can be useful for treatment planning. Additionally, for further explainability of the results, the activation map generated by the hybrid MR-U-Net has been studied.

Integrating Edges into U-Net Models with Explainable Activation Maps for Brain Tumor Segmentation using MR Images

TL;DR

This work addresses automatic brain tumor segmentation from multi-modal MRIs by proposing to learn tumor edges as additional training targets. Edges are derived from ground truth using a 3D Laplacian like filter and fused with conventional tumor region labels through one hot representations, enabling edge aware supervision across multiple U-Net variants. Across experiments on BraTS2020, edge-target training improves core tumor and edge delineation metrics and yields interpretable edge maps and activation maps, bringing classic architectures closer to state-of-the-art models like Swin U-Net and Hybrid MR-U-Net. The approach offers a simple, data oriented way to enhance segmentation precision and provide clinically useful edge information for planning and intervention.

Abstract

Manual delineation of tumor regions from magnetic resonance (MR) images is time-consuming, requires an expert, and is prone to human error. In recent years, deep learning models have been the go-to approach for the segmentation of brain tumors. U-Net and its' variants for semantic segmentation of medical images have achieved good results in the literature. However, U-Net and its' variants tend to over-segment tumor regions and may not accurately segment the tumor edges. The edges of the tumor are as important as the tumor regions for accurate diagnosis, surgical precision, and treatment planning. In the proposed work, the authors aim to extract edges from the ground truth using a derivative-like filter followed by edge reconstruction to obtain an edge ground truth in addition to the brain tumor ground truth. Utilizing both ground truths, the author studies several U-Net and its' variant architectures with and without tumor edges ground truth as a target along with the tumor ground truth for brain tumor segmentation. The author used the BraTS2020 benchmark dataset to perform the study and the results are tabulated for the dice and Hausdorff95 metrics. The mean and median metrics are calculated for the whole tumor (WT), tumor core (TC), and enhancing tumor (ET) regions. Compared to the baseline U-Net and its variants, the models that learned edges along with the tumor regions performed well in core tumor regions in both training and validation datasets. The improved performance of edge-trained models trained on baseline models like U-Net and V-Net achieved performance similar to baseline state-of-the-art models like Swin U-Net and hybrid MR-U-Net. The edge-target trained models are capable of generating edge maps that can be useful for treatment planning. Additionally, for further explainability of the results, the activation map generated by the hybrid MR-U-Net has been studied.
Paper Structure (21 sections, 7 equations, 8 figures, 5 tables, 1 algorithm)

This paper contains 21 sections, 7 equations, 8 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Proposed Methodology
  4. Z-Score Normalization
  5. Edge Extraction
  6. One-Hot Representation
  7. Training Deep Learning Models
  8. U-Net
  9. V-Net
  10. Attention U-Net
  11. U-Net 3Plus
  12. Multi Resolution U-Net with Residual Dual Attention and Deep Supervision (MRA-U-Net with RDAD)
  13. Swin U-Net
  14. Experimental Results
  15. Dataset
  16. ...and 6 more sections

Figures (8)

  • Figure 1: Shows a sample 2D axial input MR Images (a-c) and the corresponding ground truth (d). In the ground truth, the white region corresponds to the enhancing tumor (ET), the dark grey region corresponds to the necrotic core region and non-enhancing tumor (NCR/NET), and the light grey area represents the edema region (ED) sahayam2022brain.
  • Figure 2: Shows the flow diagram of the proposed methodology for training U-Net models with the ground truth and the edges as targets.
  • Figure 3: Shows the sample set of 2D axial slice input images (FLAIR, T1ce, T2) before (a-c) and the corresponding MR images (FLAIR, T1ce, T2) after (d-f) Z-score normalization for a patient in BraTS2020 dataset sahayam2022brain
  • Figure 4: 3D filter used to convolve over the ground truth to obtain the 3D edge mask.
  • Figure 5: Edge mask generated using the convolution operation on the ground truth segmentation mask of a patient followed by edge reconstruction.
  • ...and 3 more figures