Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-task Learning To Improve Semantic Segmentation Of CBCT Scans Using Image Reconstruction

Maximilian Ernst Tschuchnig, Julia Coste-Marin, Philipp Steininger, Michael Gadermayr

TL;DR

This paper addresses improving semantic segmentation of CBCT scans by leveraging multi-task learning with an image reconstruction task to regularize morphology. It extends a 3D nn-Unet baseline with a reconstruction branch and explores two strategies: mt-c (reconstruct current volume) and mt-b (reconstruct best-quality volume) using LiTS-derived CBCTs at multiple projection counts to simulate varying image quality. Results show that multi-task learning improves segmentation, particularly in patch-based setups, and the mt-b denoising variant provides additional gains, highlighting practical benefits for liver and liver tumor segmentation under degraded CBCT acquisitions. The work demonstrates the potential of joint segmentation-reconstruction training to improve performance under low-dose CBCT conditions.

Abstract

Semantic segmentation is a crucial task in medical image processing, essential for segmenting organs or lesions such as tumors. In this study we aim to improve automated segmentation in CBCTs through multi-task learning. To evaluate effects on different volume qualities, a CBCT dataset is synthesised from the CT Liver Tumor Segmentation Benchmark (LiTS) dataset. To improve segmentation, two approaches are investigated. First, we perform multi-task learning to add morphology based regularization through a volume reconstruction task. Second, we use this reconstruction task to reconstruct the best quality CBCT (most similar to the original CT), facilitating denoising effects. We explore both holistic and patch-based approaches. Our findings reveal that, especially using a patch-based approach, multi-task learning improves segmentation in most cases and that these results can further be improved by our denoising approach.

Multi-task Learning To Improve Semantic Segmentation Of CBCT Scans Using Image Reconstruction

TL;DR

This paper addresses improving semantic segmentation of CBCT scans by leveraging multi-task learning with an image reconstruction task to regularize morphology. It extends a 3D nn-Unet baseline with a reconstruction branch and explores two strategies: mt-c (reconstruct current volume) and mt-b (reconstruct best-quality volume) using LiTS-derived CBCTs at multiple projection counts to simulate varying image quality. Results show that multi-task learning improves segmentation, particularly in patch-based setups, and the mt-b denoising variant provides additional gains, highlighting practical benefits for liver and liver tumor segmentation under degraded CBCT acquisitions. The work demonstrates the potential of joint segmentation-reconstruction training to improve performance under low-dose CBCT conditions.

Abstract

Semantic segmentation is a crucial task in medical image processing, essential for segmenting organs or lesions such as tumors. In this study we aim to improve automated segmentation in CBCTs through multi-task learning. To evaluate effects on different volume qualities, a CBCT dataset is synthesised from the CT Liver Tumor Segmentation Benchmark (LiTS) dataset. To improve segmentation, two approaches are investigated. First, we perform multi-task learning to add morphology based regularization through a volume reconstruction task. Second, we use this reconstruction task to reconstruct the best quality CBCT (most similar to the original CT), facilitating denoising effects. We explore both holistic and patch-based approaches. Our findings reveal that, especially using a patch-based approach, multi-task learning improves segmentation in most cases and that these results can further be improved by our denoising approach.
Paper Structure (6 sections, 3 figures)

This paper contains 6 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Method
  3. Dataset
  4. Experimental Setup
  5. Results
  6. Discussion

Figures (3)

  • Figure 1.1: Segmentation unet (red path) and the reconstruction task (blue), facilitating multi-task learning. The volumes are used in patches and holistically (downscaled by $2$). The multi-task trained model performs segmentation and reconstruction at once (red and blue paths). The final loss includes Dice, BCE and L2 to facilitate training both segmentation and reconstruction.
  • Figure 1.2: Holistic segmentation results using different quality levels based on the amount of projections (boxplot x-axis). The $3$ upper boxplots show the results of baseline (nn-unet), mt-c and mt-b. The top row shows Dice scores based on liver segmentation. The bottom row shows liver tumor Dice scores. The orange lines show the median and green triangles the mean.
  • Figure 1.3: Patch-based segmentation results using different quality levels based on the amount of projections (boxplot x-axis). The $3$ upper boxplots show the results of our baseline (nn-unet), mt-c and mt-b. The top row shows Dice scores based on liver segmentation. The bottom row shows liver tumor Dice scores. The orange lines show the median and green triangles the mean.