Large-Scale Multi-Center CT and MRI Segmentation of Pancreas with Deep Learning

TL;DR

A new pancreas segmentation method, called PanSegNet, is developed, combining the strengths of nnUNet and a Transformer network with a new linear attention module enabling volumetric computation, enabling volumetric computation in cross-modality settings.

Abstract

Automated volumetric segmentation of the pancreas on cross-sectional imaging is needed for diagnosis and follow-up of pancreatic diseases. While CT-based pancreatic segmentation is more established, MRI-based segmentation methods are understudied, largely due to a lack of publicly available datasets, benchmarking research efforts, and domain-specific deep learning methods. In this retrospective study, we collected a large dataset (767 scans from 499 participants) of T1-weighted (T1W) and T2-weighted (T2W) abdominal MRI series from five centers between March 2004 and November 2022. We also collected CT scans of 1,350 patients from publicly available sources for benchmarking purposes. We developed a new pancreas segmentation method, called PanSegNet, combining the strengths of nnUNet and a Transformer network with a new linear attention module enabling volumetric computation. We tested PanSegNet's accuracy in cross-modality (a total of 2,117 scans) and cross-center settings with Dice and Hausdorff distance (HD95) evaluation metrics. We used Cohen's kappa statistics for intra and inter-rater agreement evaluation and paired t-tests for volume and Dice comparisons, respectively. For segmentation accuracy, we achieved Dice coefficients of 88.3% (std: 7.2%, at case level) with CT, 85.0% (std: 7.9%) with T1W MRI, and 86.3% (std: 6.4%) with T2W MRI. There was a high correlation for pancreas volume prediction with R^2 of 0.91, 0.84, and 0.85 for CT, T1W, and T2W, respectively. We found moderate inter-observer (0.624 and 0.638 for T1W and T2W MRI, respectively) and high intra-observer agreement scores. All MRI data is made available at https://osf.io/kysnj/. Our source code is available at https://github.com/NUBagciLab/PaNSegNet.

Figures (9)

• Figure 1: Flowchart showing the determination of the final study population. We select Center #1 and Center #2 data centers as in-distribution centers (internal validation) for five cross-fold training and Center #3, Center #4, and Center #5 as out-of-distribution centers (external validation). Center#1:, Center#2:, Center#3:, Center#4:, Center#5:
• Figure 2: PanSegNet is based on a combination of nnUnet with linear self-attention mechanism. Linear self-attention is obtained by converting the self-attention mechanism with linearization operation, as described below. The architecture accepts volumetric input, therefore appreciating the full anatomy details compared to pseudo-3D approaches.
• Figure 3: Comparison of traditional self-attention mechanism (left) v.s. linear self-attention mechanism (right). $X$ is input, $O$ is output. Red fonts show the specific changes we apply to self-attention to linearize.
• Figure 4: Segmentation results for CT pancreas across multiple datasets (green indicates the predicted pancreas, and red indicates the annotations). While AbdominalCT-1K exhibits robust segmentation performance, marked by precise boundary delineation, a domain shift is observed when extending the model to the AMOS, WORD, and BTCV datasets, underscoring the significance of addressing domain shifts for clinical applications. For a fair comparison, we select the visualization samples near the median value according to the Dice coefficient distribution (note: Dice is calculated volumetrically).
• Figure 5: MRI T1W pancreas segmentation visualization across various data centers. The segmentation delineations illustrate the model's capability to delineate pancreas boundaries precisely, exemplified by the accurate results. We observe domain shifts in external validation from Centers #3, #4, and #5.