3D Vascular Segmentation Supervised by 2D Annotation of Maximum Intensity Projection

TL;DR

This work tackles the costly annotation burden in 3D vascular segmentation by introducing a weakly supervised framework that leverages 2D maximum intensity projection (MIP) annotations to supervise 3D vessel segmentation. It generates 3D pseudo-labels from 2D MIP, then trains a 2D-3D fusion network that integrates 2D MIP cues with 3D volumetric data. Confidence learning and uncertainty estimation refine the pseudo-labels, followed by joint optimization with a dual-loss objective, achieving near fully-supervised performance across five datasets while dramatically reducing annotation time. The approach demonstrates strong generalization, effective ablations, and practical potential for scalable vascular analysis in clinical workflows.

Abstract

Vascular structure segmentation plays a crucial role in medical analysis and clinical applications. The practical adoption of fully supervised segmentation models is impeded by the intricacy and time-consuming nature of annotating vessels in the 3D space. This has spurred the exploration of weakly-supervised approaches that reduce reliance on expensive segmentation annotations. Despite this, existing weakly supervised methods employed in organ segmentation, which encompass points, bounding boxes, or graffiti, have exhibited suboptimal performance when handling sparse vascular structure. To alleviate this issue, we employ maximum intensity projection (MIP) to decrease the dimensionality of 3D volume to 2D image for efficient annotation, and the 2D labels are utilized to provide guidance and oversight for training 3D vessel segmentation model. Initially, we generate pseudo-labels for 3D blood vessels using the annotations of 2D projections. Subsequently, taking into account the acquisition method of the 2D labels, we introduce a weakly-supervised network that fuses 2D-3D deep features via MIP to further improve segmentation performance. Furthermore, we integrate confidence learning and uncertainty estimation to refine the generated pseudo-labels, followed by fine-tuning the segmentation network. Our method is validated on five datasets (including cerebral vessel, aorta and coronary artery), demonstrating highly competitive performance in segmenting vessels and the potential to significantly reduce the time and effort required for vessel annotation. Our code is available at: https://github.com/gzq17/Weakly-Supervised-by-MIP.

Figures (10)

• Figure 1: \ref{['pic1_a']} is an original 3D image. The characteristics of vessels on 2D sections are ambiguous and scattered, as shown in \ref{['pic1_b']}. \ref{['pic1_c']} is the MIP image of \ref{['pic1_a']}. Compared with \ref{['pic1_a']}, annotating vessels in MIP image is obviously much easier.
• Figure 2: The proposed weakly-supervised vessel segmentation framework. Initially, we employ MIP to reduce dimensionality of 3D volume for annotating. Subsequently, we introduce a novel 2D-3D feature fusion network, which is trained with pseudo labels generated from 2D annotations. To enhance the efficacy of the network, we integrate confidence learning and uncertainty estimation methods to refine the pseudo labels, followed by fine-tuning of the network.
• Figure 3: Preprocessing of Coronary CTA and Aorta CTA datasets: \ref{['pic3_a']} and \ref{['pic3_d']} depict a slice of the coronary and aorta volume, respectively. \ref{['pic3_b']} and \ref{['pic3_e']} show the MIP images obtained through direct projection, revealing that a significant portion of vessels is obscured by other anatomical structures. The MIP images of processed volumes are displayed in \ref{['pic3_c']} and \ref{['pic3_f']}, with a clear display of the majority of the blood vessel.
• Figure 4: Annotations of different categories.
• Figure 5: Segmentation results on three testing images from three datasets (TubeTK, Coronary CTA, Aorta CTA dataset in order). The red boxes highlight close-ups of some vessels.