Architecture Analysis and Benchmarking of 3D U-shaped Deep Learning Models for Thoracic Anatomical Segmentation

TL;DR

This study tackles the problem of accurate and efficient 3D thoracic anatomical segmentation for surgical planning by benchmarking six 3D U-shaped architectures (3DUNet, STUNet, AttentionUNet, SwinUNETR, FocalSegNet, and 3DSwinUnet) and four variants of 3DSwinUnet on the TotalSegmentator BTCV-task dataset. It systematically analyzes the effects of attention mechanisms, the number of resolution stages, and specific architectural components, using Dice and NSD metrics along with model complexity and latency, and employs a formal algorithm ranking framework. The key findings show that STUNet generally achieves the best trade-off between accuracy and efficiency, while 3DSwinUnet variants can improve with residual skip connections and carefully chosen upsampling, though transformer-heavy models do not consistently outperform CNN baselines in this setting. The work provides practical guidance for clinical deployment and future design, including open-source code and pretrained weights to facilitate replication and adoption.

Abstract

Recent rising interests in patient-specific thoracic surgical planning and simulation require efficient and robust creation of digital anatomical models from automatic medical image segmentation algorithms. Deep learning (DL) is now state-of-the-art in various radiological tasks, and U-shaped DL models have particularly excelled in medical image segmentation since the inception of the 2D UNet. To date, many variants of U-shaped models have been proposed by the integration of different attention mechanisms and network configurations. Systematic benchmark studies which analyze the architecture of these models by leveraging the recent development of the multi-label databases, can provide valuable insights for clinical deployment and future model designs, but such studies are still rare. We conduct the first systematic benchmark study for variants of 3D U-shaped models (3DUNet, STUNet, AttentionUNet, SwinUNETR, FocalSegNet, and a novel 3D SwinUnet with four variants) with a focus on CT-based anatomical segmentation for thoracic surgery. Our study systematically examines the impact of different attention mechanisms, the number of resolution stages, and network configurations on segmentation accuracy and computational complexity. To allow cross-reference with other recent benchmarking studies, we also included a performance assessment of the BTCV abdominal structural segmentation. With the STUNet ranking at the top, our study demonstrated the value of CNN-based U-shaped models for the investigated tasks and the benefit of residual blocks in network configuration designs to boost segmentation performance.

Figures (1)

• Figure 1: Demonstration of the anatomical structures for U-shaped model benchmarking, including 12 labels for thoracic surgery and 13 labels that are consistent with the BTCV segmentation challenge.