Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Size Aware Cross-shape Scribble Supervision for Medical Image Segmentation

Jing Yuan, Tania Stathaki

TL;DR

The paper tackles the high labeling cost and inconsistency of scribble supervision in medical image segmentation by introducing cross-shape scribble annotations, a pseudo mask generation strategy, and a size-aware two-stage multi-branch network. The PMG module converts cross-shape scribbles into informative pseudo masks, while the TMB module learns branch-specific segmentation for small, medium, and large targets and selects the best branch through a learned confidence score. Key contributions include (i) cross-shape scribbling for robust, fast labeling; (ii) a pseudo-mask guided supervision that accelerates learning; and (iii) a size-aware loss and multi-branch framework that improves segmentation across scales, achieving state-of-the-art results on multiple polyp datasets and generalizing to the ACDC dataset. Collectively, these methods reduce annotation effort and improve accuracy for scale-variant medical segmentation, with practical impact for faster, cost-effective clinical analysis.

Abstract

Scribble supervision, a common form of weakly supervised learning, involves annotating pixels using hand-drawn curve lines, which helps reduce the cost of manual labelling. This technique has been widely used in medical image segmentation tasks to fasten network training. However, scribble supervision has limitations in terms of annotation consistency across samples and the availability of comprehensive groundtruth information. Additionally, it often grapples with the challenge of accommodating varying scale targets, particularly in the context of medical images. In this paper, we propose three novel methods to overcome these challenges, namely, 1) the cross-shape scribble annotation method; 2) the pseudo mask method based on cross shapes; and 3) the size-aware multi-branch method. The parameter and structure design are investigated in depth. Experimental results show that the proposed methods have achieved significant improvement in mDice scores across multiple polyp datasets. Notably, the combination of these methods outperforms the performance of state-of-the-art scribble supervision methods designed for medical image segmentation.

Size Aware Cross-shape Scribble Supervision for Medical Image Segmentation

TL;DR

The paper tackles the high labeling cost and inconsistency of scribble supervision in medical image segmentation by introducing cross-shape scribble annotations, a pseudo mask generation strategy, and a size-aware two-stage multi-branch network. The PMG module converts cross-shape scribbles into informative pseudo masks, while the TMB module learns branch-specific segmentation for small, medium, and large targets and selects the best branch through a learned confidence score. Key contributions include (i) cross-shape scribbling for robust, fast labeling; (ii) a pseudo-mask guided supervision that accelerates learning; and (iii) a size-aware loss and multi-branch framework that improves segmentation across scales, achieving state-of-the-art results on multiple polyp datasets and generalizing to the ACDC dataset. Collectively, these methods reduce annotation effort and improve accuracy for scale-variant medical segmentation, with practical impact for faster, cost-effective clinical analysis.

Abstract

Scribble supervision, a common form of weakly supervised learning, involves annotating pixels using hand-drawn curve lines, which helps reduce the cost of manual labelling. This technique has been widely used in medical image segmentation tasks to fasten network training. However, scribble supervision has limitations in terms of annotation consistency across samples and the availability of comprehensive groundtruth information. Additionally, it often grapples with the challenge of accommodating varying scale targets, particularly in the context of medical images. In this paper, we propose three novel methods to overcome these challenges, namely, 1) the cross-shape scribble annotation method; 2) the pseudo mask method based on cross shapes; and 3) the size-aware multi-branch method. The parameter and structure design are investigated in depth. Experimental results show that the proposed methods have achieved significant improvement in mDice scores across multiple polyp datasets. Notably, the combination of these methods outperforms the performance of state-of-the-art scribble supervision methods designed for medical image segmentation.
Paper Structure (42 sections, 11 equations, 16 figures, 11 tables)

This paper contains 42 sections, 11 equations, 16 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Overall pipeline
  5. Pseudo Mask Generation (PMG)
  6. Cross-shape scribbling
  7. Generation of the pseudo mask
  8. Advantage
  9. Two-stage Multiple Branches (TMB)
  10. Train segmentation stage
  11. Size-aware loss function
  12. Train score stage
  13. Experiments
  14. Setup
  15. Dataset and evaluation metric
  16. ...and 27 more sections

Figures (16)

  • Figure 1: Examples of relatively large and small polyps, the full groundtruth masks and the scribbles used in this paper to annotate the foreground and background area. (a) The images. (b) Full groundtruth masks. (c) Foreground scribble in crossing shape. (d) Background scribble.
  • Figure 2: (a) The example of the cross-shape scribble and the corresponding pseudo mask of a polyp. The foreground(polyp) is annotated with two nearly perpendicular line segments (green). The background is annotated with a single line segment (yellow). All the scribbles have a width of one pixel. The generated pseudo mask (grey) is the outer parallelogram of the cross-shape scribble. (b) Examples of cross-shape scribble annotations for twisted polyps used in this paper. The upper row shows the full masks. The bottom row shows the cross-shape scribble annotations. The red and green boxes mark the areas that will be ignored or mistakenly included after applying the pseudo mask generated by the rough cross-shape scribbles.
  • Figure 3: The sketch of the generation of the pseudo mask based on cross-shape scribbles. A weighted mask is initialized within the region defined by the crossing scribbles as depicted by the blue shaded area. Subsequently, this mask is rotated to align with the orientation of the target, as shown in the dotted area.
  • Figure 4: The architecture of the proposed size-aware multi-branch method. The network consists of the Pseudo Mask Generation (PMG) module and the Two-stage Multiple Branches (TMB) module. The TMB module comprises the train segmentation stage and the train score stage. The train segmentation component generates accurate prediction results through multiple branches. Subsequently, the backbone and the train segmentation component are held constant with updates applied exclusively to the parameters of the train score component. This updated component predicts confidence scores, indicating which branch is best suited to produce the optimal mask. The detailed structure of the Coefficient Mask Generation (CMG), Branch Selection (BS), GT Score Generation (GSG) and the Channel-wise Weighted Average (CWA) blocks are presented in \ref{['sec: size aware multi-branch method']}.
  • Figure 5: The cross-shape scribble and the resulting pseudo masks generated by multiplication, addition, and maximization respectively.
  • ...and 11 more figures