Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-head automated segmentation by incorporating detection head into the contextual layer neural network

Edwin Kys, Febian Febian

TL;DR

The paper tackles hallucinations in deep learning–based CT auto-segmentation for radiotherapy by introducing a gated multi-head Transformer architecture built on Swin U-Net, augmented with inter-slice context and a parallel slice-level detection head. This design decouples structure presence reasoning from pixel-level segmentation and gates segmentation predictions based on detected anatomical presence. On the Prostate-Anatomical-Edge-Cases dataset, the gated model markedly reduces false positives and improves robustness compared with a segmentation-only baseline, demonstrating enhanced anatomical plausibility for auto-contouring. This approach offers a promising direction for safer, more reliable radiotherapy workflows and lays groundwork for cross-site generalization and uncertainty-aware gating in clinical settings.

Abstract

Deep learning based auto segmentation is increasingly used in radiotherapy, but conventional models often produce anatomically implausible false positives, or hallucinations, in slices lacking target structures. We propose a gated multi-head Transformer architecture based on Swin U-Net, augmented with inter-slice context integration and a parallel detection head, which jointly performs slice-level structure detection via a multi-layer perceptron and pixel-level segmentation through a context-enhanced stream. Detection outputs gate the segmentation predictions to suppress false positives in anatomically invalid slices, and training uses slice-wise Tversky loss to address class imbalance. Experiments on the Prostate-Anatomical-Edge-Cases dataset from The Cancer Imaging Archive demonstrate that the gated model substantially outperforms a non-gated segmentation-only baseline, achieving a mean Dice loss of $0.013 \pm 0.036$ versus $0.732 \pm 0.314$, with detection probabilities strongly correlated with anatomical presence, effectively eliminating spurious segmentations. In contrast, the non-gated model exhibited higher variability and persistent false positives across all slices. These results indicate that detection-based gating enhances robustness and anatomical plausibility in automated segmentation applications, reducing hallucinated predictions without compromising segmentation quality in valid slices, and offers a promising approach for improving the reliability of clinical radiotherapy auto-contouring workflows.

Multi-head automated segmentation by incorporating detection head into the contextual layer neural network

TL;DR

The paper tackles hallucinations in deep learning–based CT auto-segmentation for radiotherapy by introducing a gated multi-head Transformer architecture built on Swin U-Net, augmented with inter-slice context and a parallel slice-level detection head. This design decouples structure presence reasoning from pixel-level segmentation and gates segmentation predictions based on detected anatomical presence. On the Prostate-Anatomical-Edge-Cases dataset, the gated model markedly reduces false positives and improves robustness compared with a segmentation-only baseline, demonstrating enhanced anatomical plausibility for auto-contouring. This approach offers a promising direction for safer, more reliable radiotherapy workflows and lays groundwork for cross-site generalization and uncertainty-aware gating in clinical settings.

Abstract

Deep learning based auto segmentation is increasingly used in radiotherapy, but conventional models often produce anatomically implausible false positives, or hallucinations, in slices lacking target structures. We propose a gated multi-head Transformer architecture based on Swin U-Net, augmented with inter-slice context integration and a parallel detection head, which jointly performs slice-level structure detection via a multi-layer perceptron and pixel-level segmentation through a context-enhanced stream. Detection outputs gate the segmentation predictions to suppress false positives in anatomically invalid slices, and training uses slice-wise Tversky loss to address class imbalance. Experiments on the Prostate-Anatomical-Edge-Cases dataset from The Cancer Imaging Archive demonstrate that the gated model substantially outperforms a non-gated segmentation-only baseline, achieving a mean Dice loss of versus , with detection probabilities strongly correlated with anatomical presence, effectively eliminating spurious segmentations. In contrast, the non-gated model exhibited higher variability and persistent false positives across all slices. These results indicate that detection-based gating enhances robustness and anatomical plausibility in automated segmentation applications, reducing hallucinated predictions without compromising segmentation quality in valid slices, and offers a promising approach for improving the reliability of clinical radiotherapy auto-contouring workflows.
Paper Structure (4 sections, 3 figures, 1 table)

This paper contains 4 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Materials and Methods
  3. Results and Discussion
  4. Conclusion

Figures (3)

  • Figure 1: Overview of N2 model workflow consisting of patch embedding, encoder layers with context integration, skip connections to the decoder, and temporal context fusion with the addition of detection head in its pipeline.
  • Figure 2: Dice loss comparison between the gated multi-head model and the non-gated segmentation-only model.
  • Figure 3: Comparison of multi-channel segmentation results for prostate, bladder, and rectum across sequential slices (45, 60, 61, and 62) from a representative case. Ground truth annotations are shown in the left column, predictions from the gated multi-head model in the middle column, and predictions from the segmentation-only model in the right column.