Can Foundation Models Really Segment Tumors? A Benchmarking Odyssey in Lung CT Imaging

TL;DR

This paper benchmarks traditional segmentation architectures (DeepLabV3, U-Net, nnUNet) against foundation-model adaptations (MedSAM, MedSAM 2) for lung tumor segmentation in CT imaging across two datasets. It explores zero-shot, few-shot, and fine-tuning regimes, highlighting that MedSAM 2 with bounding-box prompts achieves the best tumor segmentation performance with favorable efficiency, whereas traditional models struggle with tumor delineation. The study demonstrates that prompt design and data availability significantly influence foundation-model performance, and that lung segmentation remains easier than tumor segmentation. The findings suggest strong potential for foundation models in clinical workflows, while also outlining directions to automate prompts and extend benchmarking to larger, diverse datasets and federated settings.

Abstract

Accurate lung tumor segmentation is crucial for improving diagnosis, treatment planning, and patient outcomes in oncology. However, the complexity of tumor morphology, size, and location poses significant challenges for automated segmentation. This study presents a comprehensive benchmarking analysis of deep learning-based segmentation models, comparing traditional architectures such as U-Net and DeepLabV3, self-configuring models like nnUNet, and foundation models like MedSAM, and MedSAM~2. Evaluating performance across two lung tumor segmentation datasets, we assess segmentation accuracy and computational efficiency under various learning paradigms, including few-shot learning and fine-tuning. The results reveal that while traditional models struggle with tumor delineation, foundation models, particularly MedSAM~2, outperform them in both accuracy and computational efficiency. These findings underscore the potential of foundation models for lung tumor segmentation, highlighting their applicability in improving clinical workflows and patient outcomes.

Figures (2)

• Figure 1: Qualitative comparison of segmentation results across different models. The first column presents the original CT scan images, followed by the ground truth segmentations of left and right lungs and tumor mass. The remaining columns showcase the predictions generated by the benchmarking models.
• Figure 2: Comparison of segmentation models in terms of Dice Score (y-axis) and computational cost measured in GMACs (x-axis, log scale). The size of each bubble is proportional to the number of model parameters, as illustrated by the gray reference bubbles in the bottom right corner, corresponding to 10M, 50M, and 100M parameters.