SparseMamba-PCL: Scribble-Supervised Medical Image Segmentation via SAM-Guided Progressive Collaborative Learning

TL;DR

Scribble supervision reduces labeling cost but struggles to propagate sparse labels to dense masks and recover precise boundaries. We introduce Progressive Collaborative Learning, a framework that combines Scribble-Propagated Boundary Estimator, Sparse Mamba network, and Med-SAM guidance to enrich scribbles, fuse embeddings, and refine segmentations. The approach includes boundary enrichment, skip-sampling-based global dependency modeling, and iterative Med-SAM-guided refinement, optimized by a combined Dice and partial cross-entropy loss, with the Med-SAM decoder further fine-tuned. Experiments on ACDC, CHAOS, and MSCMRSeg demonstrate state-of-the-art Dice scores and robust boundary accuracy, outperforming nine SOTA scribble-based methods, with code available at https://github.com/QLYCode/SparseMamba-PCL.git.

Abstract

Scribble annotations significantly reduce the cost and labor required for dense labeling in large medical datasets with complex anatomical structures. However, current scribble-supervised learning methods are limited in their ability to effectively propagate sparse annotation labels to dense segmentation masks and accurately segment object boundaries. To address these issues, we propose a Progressive Collaborative Learning framework that leverages novel algorithms and the Med-SAM foundation model to enhance information quality during training. (1) We enrich ground truth scribble segmentation labels through a new algorithm, propagating scribbles to estimate object boundaries. (2) We enhance feature representation by optimizing Med-SAM-guided training through the fusion of feature embeddings from Med-SAM and our proposed Sparse Mamba network. This enriched representation also facilitates the fine-tuning of the Med-SAM decoder with enriched scribbles. (3) For inference, we introduce a Sparse Mamba network, which is highly capable of capturing local and global dependencies by replacing the traditional sequential patch processing method with a skip-sampling procedure. Experiments on the ACDC, CHAOS, and MSCMRSeg datasets validate the effectiveness of our framework, outperforming nine state-of-the-art methods. Our code is available at \href{https://github.com/QLYCode/SparseMamba-PCL}{SparseMamba-PCL.git}.

Figures (4)

• Figure 1: Overview of the SparseMamba-PCL framework for scribble-supervised image segmentation. (a) Scribble-Propagated Object Boundary Estimator, (b) SparseMamba, (c) Med-SAM Network, (d) Sparse Mamba Block, and (e) Dual Attention Module.
• Figure 2: Visualization of one iteration in the boundary estimation algorithm. (a) $S_i$ shows the dilated scribble and (b) $U_i$ the counting map for the boundary map $E_i$ at the current step $i$. Blue regions in (a) and (b) represent pixels, for which the operations in $S_i$ and $U_i$ hold true respectively. $S$ and $U$ are logically combined with the edge map $F$ to extract edge pixels, which are added to update the boundary map to $E_{i+1}$. The process is repeated $j$ times for all kernel sizes and for every class to generate the final boundary map $E$, as shown in (e).
• Figure 3: (a) Vmamba liu2024vmamba uses the 2D Selective Scan (SS2D) method, scanning four directions. (b) Sparse SS2D omits some sampling steps and performs intra-group traversal with a skipping step of 2.
• Figure 4: Qualitative comparison of weakly-supervised segmentation methods on ACDC, CHAOS, and MSCMRSeg datasets. (a) Input image, (b) ground truth, and segmentation results from (c) USTM liu2022weakly, (d) Scribble2D5 chen2022scribble2d5, (e) CycleMix zhang2022cyclemix, (f) ShapePU zhang2022shapepu, (g) S²ME wang2023s, (h) ScribbleVC li2023scribblevc, (i) TDNet han2023scribble, (j) PacingPseudo yang2024non, (k) Scribbleformer li2024scribformer, and (l) SparseMamba-PCL are given.