Sli2Vol+: Segmenting 3D Medical Images Based on an Object Estimation Guided Correspondence Flow Network

TL;DR

A new SSF, called Sli2Vol+, is proposed for segmenting any anatomical structures in 3D medical images using only a single annotated slice per training and testing volume, and a novel Object Estimation Guided Correspondence Flow Network is developed to learn reliable correspondences between consecutive slices and corresponding PLs in a self-supervised manner.

Abstract

Deep learning (DL) methods have shown remarkable successes in medical image segmentation, often using large amounts of annotated data for model training. However, acquiring a large number of diverse labeled 3D medical image datasets is highly difficult and expensive. Recently, mask propagation DL methods were developed to reduce the annotation burden on 3D medical images. For example, Sli2Vol~\cite{yeung2021sli2vol} proposed a self-supervised framework (SSF) to learn correspondences by matching neighboring slices via slice reconstruction in the training stage; the learned correspondences were then used to propagate a labeled slice to other slices in the test stage. But, these methods are still prone to error accumulation due to the inter-slice propagation of reconstruction errors. Also, they do not handle discontinuities well, which can occur between consecutive slices in 3D images, as they emphasize exploiting object continuity. To address these challenges, in this work, we propose a new SSF, called \proposed, {for segmenting any anatomical structures in 3D medical images using only a single annotated slice per training and testing volume.} Specifically, in the training stage, we first propagate an annotated 2D slice of a training volume to the other slices, generating pseudo-labels (PLs). Then, we develop a novel Object Estimation Guided Correspondence Flow Network to learn reliable correspondences between consecutive slices and corresponding PLs in a self-supervised manner. In the test stage, such correspondences are utilized to propagate a single annotated slice to the other slices of a test volume. We demonstrate the effectiveness of our method on various medical image segmentation tasks with different datasets, showing better generalizability across different organs, modalities, and modals. Code is available at \url{https://github.com/adlsn/Sli2Volplus}

Figures (4)

• Figure 1: The pipeline of our proposed framework. (a) Training stage: Adjacent 2D slices and their corresponding generated pseudo-labels (PLs) are sampled from a 3D volume to train the Object Estimation Guided Correspondence Flow Network (OEG-CFN). (b) The architecture of OEG-CFN. (c) Test stage: The trained OEG-CFN is used to propagate a labeled slice to the other slices of the entire volume (five slices of a test volume are shown in the example). Red annotations represent ground truth segmentations, yellow and pink annotations represent PLs, and orange annotations represent the final segmentations. $\bigotimes$ denotes matrix multiplication.
• Figure 2: Examples of segmentation results by different methods on the Decath-Liver (top) and Decath-Spleen (bottom) datasets simpson2019large. In the examples, a labeled slice is propagated in two directions, and the "$i$-th" represents the position of a slice to which the labeled slice is propagated in that direction.
• Figure 3: Examples of segmentation results by various segmentation foundation models or interactive segmentation tools on the Decath-Brain Tumours dataset simpson2019large.
• Figure 4: Examples of segmentation results generated by our method when combining different key components on the Decath-Brain Tumours and Heart datasets simpson2019large.