SegAnyPET: Universal Promptable Segmentation from Positron Emission Tomography Images

Abstract

Positron Emission Tomography (PET) is a powerful molecular imaging tool that plays a crucial role in modern medical diagnostics by visualizing radio-tracer distribution to reveal physiological processes. Accurate organ segmentation from PET images is essential for comprehensive multi-systemic analysis of interactions between different organs and pathologies. Existing segmentation methods are limited by insufficient annotation data and varying levels of annotation, resulting in weak generalization ability and difficulty in clinical application. Recent developments in segmentation foundation models have shown superior versatility across diverse segmentation tasks. Despite the efforts of medical adaptations, these works primarily focus on structural medical images with detailed physiological structural information and exhibit limited generalization performance on molecular PET imaging. In this paper, we collect and construct PETS-5k, the largest PET segmentation dataset to date, comprising 5,731 three-dimensional whole-body PET images and encompassing over 1.3M 2D images. Based on the established dataset, we develop SegAnyPET, a modality-specific 3D foundation model for universal promptable segmentation from PET images. To issue the challenge of discrepant annotation quality, we adopt a cross prompting confident learning (CPCL) strategy with an uncertainty-guided self-rectification process to robustly learn segmentation from high-quality labeled data and low-quality noisy labeled data for promptable segmentation. Experimental results demonstrate that SegAnyPET can segment seen and unseen target organs using only one or a few prompt points, outperforming state-of-the-art foundation models and task-specific fully supervised models with higher accuracy and strong generalization ability for universal segmentation.

Figures (8)

• Figure 1: Visual comparison of an example case between molecular PET image and structural CT image.
• Figure 2: An overview of our work for PET segmentation foundation model. Firstly, we collect and construct PETS-5k, a large-scale PET segmentation dataset for developing the foundation model. Based on the principle of promptable segmentation, we develop SegAnyPET, a modality-specific 3D segmentation foundation model that can be efficiently adapted for universal segmentation of any target organs or lesions based on positional prompting from 3D PET images.
• Figure 3: Illustration of the proposed cross prompting confident learning (CPCL) strategy for developing promptable segmentation foundation model based on both high quality and low quality annotations.
• Figure 4: Visual comparison of segmentation results of different promptable segmentation foundation models for PET organ segmentation.
• Figure 5: Visualization of PET images and corresponding organ annotations of PETS-5k dataset and AutoPET-Organ dataset.