Intelligent Diagnosis Using Dual-Branch Attention Network for Rare Thyroid Carcinoma Recognition with Ultrasound Imaging

TL;DR

A novel multitask learning framework, Channel-Spatial Attention Synergy Network (CSASN), which integrates a dual-branch feature extractor - combining EfficientNet for local spatial encoding and ViT for global semantic modeling, with a cascaded channel-spatial attention refinement module to enhance classification stability and accuracy.

Abstract

Heterogeneous morphological features and data imbalance pose significant challenges in rare thyroid carcinoma classification using ultrasound imaging. To address this issue, we propose a novel multitask learning framework, Channel-Spatial Attention Synergy Network (CSASN), which integrates a dual-branch feature extractor - combining EfficientNet for local spatial encoding and ViT for global semantic modeling, with a cascaded channel-spatial attention refinement module. A residual multiscale classifier and dynamically weighted loss function further enhance classification stability and accuracy. Trained on a multicenter dataset comprising more than 2000 patients from four clinical institutions, our framework leverages a residual multiscale classifier and dynamically weighted loss function to enhance classification stability and accuracy. Extensive ablation studies demonstrate that each module contributes significantly to model performance, particularly in recognizing rare subtypes such as FTC and MTC carcinomas. Experimental results show that CSASN outperforms existing single-stream CNN or Transformer-based models, achieving a superior balance between precision and recall under class-imbalanced conditions. This framework provides a promising strategy for AI-assisted thyroid cancer diagnosis.

Figures (7)

• Figure 1: An example of our dataset, including benign nodule, and 3 subtypes: ATC, FTC and MTC. ’M’ means malignant and they may come from different centers.
• Figure 2: Flow chart of this study, including Data Acquisition and Preprocessing, Dual Modal Cooperative Feature Extraction, Cascaded Attention Refinement, and Residual Multiscale Classification.
• Figure 3: ROC curves comparing CSASN with baseline models across thyroid carcinoma subtypes: (A) ATC, (B) FTC, (C) MTC. Top row: CNN-based models; bottom row: Transformer-based models.
• Figure 4: Percentage improvement in AUC of CSASN over individual baseline models for (A) ATC, (B) FTC, and (C) MTC classification tasks.
• Figure 5: Confusion matrices for the three ablation variants. Top: ablation 1; Mid: ablation 2; Bottom: ablation 3.