Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Lung-CADex: Fully automatic Zero-Shot Detection and Classification of Lung Nodules in Thoracic CT Images

Furqan Shaukat, Syed Muhammad Anwar, Abhijeet Parida, Van Khanh Lam, Marius George Linguraru, Mubarak Shah

TL;DR

This work addresses the challenge of automatic, lesion-level detection and malignancy classification of lung nodules in CT images under limited labeled data. It introduces a two-stage, end-to-end pipeline: CADe uses a zero-shot MedSAM segmentation guided by a textual prompt suite (with prefix-tuned text encoders) to detect nodules, while CADx employs a CLIP-inspired framework that aligns segmented nodule patches with a radiomic-feature gallery using a ResNet50 image encoder, followed by a binary benign/malignant classifier. Training leverages the LIDC dataset (and the LUNA subset) and evaluates generalization on the LUNGx dataset, achieving a sensitivity of 0.86 and competitive AUC/accuracy metrics compared to fully supervised baselines, including ablations that reveal a five-feature (k=5) configuration as optimal for sensitivity. The approach demonstrates a practical, fully automatic end-to-end solution for lung cancer screening, with potential for clinical deployment and integration with electronic medical records, while acknowledging data limitations and the benefits of richer annotations.

Abstract

Lung cancer has been one of the major threats to human life for decades. Computer-aided diagnosis can help with early lung nodul detection and facilitate subsequent nodule characterization. Large Visual Language models (VLMs) have been found effective for multiple downstream medical tasks that rely on both imaging and text data. However, lesion level detection and subsequent diagnosis using VLMs have not been explored yet. We propose CADe, for segmenting lung nodules in a zero-shot manner using a variant of the Segment Anything Model called MedSAM. CADe trains on a prompt suite on input computed tomography (CT) scans by using the CLIP text encoder through prefix tuning. We also propose, CADx, a method for the nodule characterization as benign/malignant by making a gallery of radiomic features and aligning image-feature pairs through contrastive learning. Training and validation of CADe and CADx have been done using one of the largest publicly available datasets, called LIDC. To check the generalization ability of the model, it is also evaluated on a challenging dataset, LUNGx. Our experimental results show that the proposed methods achieve a sensitivity of 0.86 compared to 0.76 that of other fully supervised methods.The source code, datasets and pre-processed data can be accessed using the link:

Lung-CADex: Fully automatic Zero-Shot Detection and Classification of Lung Nodules in Thoracic CT Images

TL;DR

This work addresses the challenge of automatic, lesion-level detection and malignancy classification of lung nodules in CT images under limited labeled data. It introduces a two-stage, end-to-end pipeline: CADe uses a zero-shot MedSAM segmentation guided by a textual prompt suite (with prefix-tuned text encoders) to detect nodules, while CADx employs a CLIP-inspired framework that aligns segmented nodule patches with a radiomic-feature gallery using a ResNet50 image encoder, followed by a binary benign/malignant classifier. Training leverages the LIDC dataset (and the LUNA subset) and evaluates generalization on the LUNGx dataset, achieving a sensitivity of 0.86 and competitive AUC/accuracy metrics compared to fully supervised baselines, including ablations that reveal a five-feature (k=5) configuration as optimal for sensitivity. The approach demonstrates a practical, fully automatic end-to-end solution for lung cancer screening, with potential for clinical deployment and integration with electronic medical records, while acknowledging data limitations and the benefits of richer annotations.

Abstract

Lung cancer has been one of the major threats to human life for decades. Computer-aided diagnosis can help with early lung nodul detection and facilitate subsequent nodule characterization. Large Visual Language models (VLMs) have been found effective for multiple downstream medical tasks that rely on both imaging and text data. However, lesion level detection and subsequent diagnosis using VLMs have not been explored yet. We propose CADe, for segmenting lung nodules in a zero-shot manner using a variant of the Segment Anything Model called MedSAM. CADe trains on a prompt suite on input computed tomography (CT) scans by using the CLIP text encoder through prefix tuning. We also propose, CADx, a method for the nodule characterization as benign/malignant by making a gallery of radiomic features and aligning image-feature pairs through contrastive learning. Training and validation of CADe and CADx have been done using one of the largest publicly available datasets, called LIDC. To check the generalization ability of the model, it is also evaluated on a challenging dataset, LUNGx. Our experimental results show that the proposed methods achieve a sensitivity of 0.86 compared to 0.76 that of other fully supervised methods.The source code, datasets and pre-processed data can be accessed using the link:
Paper Structure (11 sections, 4 equations, 3 figures, 2 tables)

This paper contains 11 sections, 4 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Dataset Curation and Pre-Processing
  4. Network Architecture
  5. Nodule Detection (CADe):
  6. Nodule Classification (CADx):
  7. Loss Function
  8. Nodule Segmentation:
  9. Nodule Classification:
  10. Experimental Results and Discussion
  11. Conclusions

Figures (3)

  • Figure 1: Flow chart of the proposed method which consists of two stages namely $CAD_{e}$ which refers to the detection phase and $CAD_{x}$ which refers to the diagnosis phase.
  • Figure 2: Detailed architecture of the lung nodule classification method.
  • Figure 3: Few examples of zero-shot segmentation results generated using the segmentation model and corresponding textual prompts. Green represents the segmented regions, and white represents the ground truth.