Clustering Propagation for Universal Medical Image Segmentation

TL;DR

S2VNet is introduced, a universal framework that leverages Slice-to- Volume propagation to unify auto-matic/interactive segmentation within a single model and one training session, and surpasses task-specified solutions on both automatic/interactive setups.

Abstract

Prominent solutions for medical image segmentation are typically tailored for automatic or interactive setups, posing challenges in facilitating progress achieved in one task to another.$_{\!}$ This$_{\!}$ also$_{\!}$ necessitates$_{\!}$ separate$_{\!}$ models for each task, duplicating both training time and parameters.$_{\!}$ To$_{\!}$ address$_{\!}$ above$_{\!}$ issues,$_{\!}$ we$_{\!}$ introduce$_{\!}$ S2VNet,$_{\!}$ a$_{\!}$ universal$_{\!}$ framework$_{\!}$ that$_{\!}$ leverages$_{\!}$ Slice-to-Volume$_{\!}$ propagation$_{\!}$ to$_{\!}$ unify automatic/interactive segmentation within a single model and one training session. Inspired by clustering-based segmentation techniques, S2VNet makes full use of the slice-wise structure of volumetric data by initializing cluster centers from the cluster$_{\!}$ results$_{\!}$ of$_{\!}$ previous$_{\!}$ slice.$_{\!}$ This enables knowledge acquired from prior slices to assist in the segmentation of the current slice, further efficiently bridging the communication between remote slices using mere 2D networks. Moreover, such a framework readily accommodates interactive segmentation with no architectural change, simply by initializing centroids from user inputs. S2VNet distinguishes itself by swift inference speeds and reduced memory consumption compared to prevailing 3D solutions. It can also handle multi-class interactions with each of them serving to initialize different centroids. Experiments on three benchmarks demonstrate S2VNet surpasses task-specified solutions on both automatic/interactive setups.

Figures (10)

• Figure 1: (a-b) Existing volume-wise and slice-wise solutions. (c) Our slice-to-volume solution that bridges distant slices by cluster center propagation and further unifies automatic/interactive segmentation under the same model with 2D segmentation networks.
• Figure 2: Our clustering propagation-driven universal segmentation$_{\!}$ framework$_{\!}$ (§\ref{['sec:frame']}). (a) S2VNet adapts multi-class interactive segmentation and refinement by iteratively initializing cluster centers from user clicks. (b) Our clustering-based slice-to-volume propagation pipeline where centroids are evolved during slice-level segmentation and passed to the next slices. and denote concatenation and dot product.
• Figure 3: Illustration of recurrent centroid aggregation (§\ref{['sec:frame']}). After clustering within the slice-wise segmentation for each slice, the centroids are recurrently merged with the historic ones to assist in the initialization of centroids belonging to the subsequent slice.
• Figure 4: Visual comparison results on WORD luo2022wordtest. See §\ref{['sec:vis']} for detailed analysis.
• Figure 5: Convergence analysis on WORD luo2022wordtest. We report the DSC score with different round of user interactions.