Performance of a Deep Learning-Based Segmentation Model for Pancreatic Tumors on Public Endoscopic Ultrasound Datasets

TL;DR

Pancreatic tumor segmentation in endoscopic ultrasound (EUS) is hindered by operator variability and image noise. The authors evaluate a Vision Transformer–based segmentation model within the USFM framework, trained on $n=17{,}367$ publicly available EUS images and tested on an external set of $n=350$ images from LEP, using 5-fold cross-validation. The model achieves a Dice similarity coefficient of approximately $0.658$ in cross-validation and $0.657$ on external data, with high specificity ($ ext{Sp} ightarrow ext{≈}0.988$) and moderate sensitivity ($ ext{Se} ightarrow ext{≈}0.698$–$0.718$), but 9.7\% of cases show multiple erroneous predictions, underscoring failure modes for small or poorly defined lesions. The results support the robustness of ViT-based segmentation for pancreatic tumors in EUS while highlighting the need for standardized, multicenter datasets and prospective validation to support clinical deployment.

Abstract

Background: Pancreatic cancer is one of the most aggressive cancers, with poor survival rates. Endoscopic ultrasound (EUS) is a key diagnostic modality, but its effectiveness is constrained by operator subjectivity. This study evaluates a Vision Transformer-based deep learning segmentation model for pancreatic tumors. Methods: A segmentation model using the USFM framework with a Vision Transformer backbone was trained and validated with 17,367 EUS images (from two public datasets) in 5-fold cross-validation. The model was tested on an independent dataset of 350 EUS images from another public dataset, manually segmented by radiologists. Preprocessing included grayscale conversion, cropping, and resizing to 512x512 pixels. Metrics included Dice similarity coefficient (DSC), intersection over union (IoU), sensitivity, specificity, and accuracy. Results: In 5-fold cross-validation, the model achieved a mean DSC of 0.651 +/- 0.738, IoU of 0.579 +/- 0.658, sensitivity of 69.8%, specificity of 98.8%, and accuracy of 97.5%. For the external validation set, the model achieved a DSC of 0.657 (95% CI: 0.634-0.769), IoU of 0.614 (95% CI: 0.590-0.689), sensitivity of 71.8%, and specificity of 97.7%. Results were consistent, but 9.7% of cases exhibited erroneous multiple predictions. Conclusions: The Vision Transformer-based model demonstrated strong performance for pancreatic tumor segmentation in EUS images. However, dataset heterogeneity and limited external validation highlight the need for further refinement, standardization, and prospective studies.

Figures (2)

• Figure 1: Segmentation visualization of pancreatic cancer in 2 patients with excellent segmentation by the DL model. Each row shows (left) input EUS image, (middle) ground-truth mask, (right) model prediction overlay (red). The DSC of the 1st patient (upper row) was 0.891 and that of the 2nd patient (lower row) was 0.905.
• Figure 2: Segmentation visualization of pancreatic cancer in 2 patients with lower performance of the DL model. Each row shows (left) input EUS image, (middle) ground-truth mask, (right) model prediction overlay (red). In 1st patient (upper row) the mass has ill-defined boarders (compared with the case shown in Figure 1). The DSC in this image was 0.491. In the 2nd patient (lower row) the mass show subtle difference in the echogenicity compared to the background focally and has ill-defined boarders at places leading to the low Dice score (DSC: 0.413).