Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DSCENet: Dynamic Screening and Clinical-Enhanced Multimodal Fusion for MPNs Subtype Classification

Yuan Zhang, Yaolei Qi, Xiaoming Qi, Yongyue Wei, Guanyu Yang

TL;DR

This work tackles the challenging problem of classifying myeloproliferative neoplasm subtypes by leveraging multimodal data from whole slide images and clinical information. The authors introduce DSCENet, which combines a Dynamic Screening Module for adaptive patch selection with a Clinical-enhanced Fusion Module that uses cross-modal attention to fuse image and clinical features. On a multicenter dataset of 383 cases, DSCENet achieves an ACC of 83.12% and an AUC around 96%, significantly outperforming state-of-the-art single-modality and fusion methods and demonstrating the value of dynamic patch screening and clinical guidance. Ablation studies corroborate the contributions of both modules, with clinical data providing substantial gains in AUC and patch screening improving feature representativeness, indicating strong potential for improving diagnostic accuracy and informing treatment planning in MPNs.

Abstract

The precise subtype classification of myeloproliferative neoplasms (MPNs) based on multimodal information, which assists clinicians in diagnosis and long-term treatment plans, is of great clinical significance. However, it remains a great challenging task due to the lack of diagnostic representativeness for local patches and the absence of diagnostic-relevant features from a single modality. In this paper, we propose a Dynamic Screening and Clinical-Enhanced Network (DSCENet) for the subtype classification of MPNs on the multimodal fusion of whole slide images (WSIs) and clinical information. (1) A dynamic screening module is proposed to flexibly adapt the feature learning of local patches, reducing the interference of irrelevant features and enhancing their diagnostic representativeness. (2) A clinical-enhanced fusion module is proposed to integrate clinical indicators to explore complementary features across modalities, providing comprehensive diagnostic information. Our approach has been validated on the real clinical data, achieving an increase of 7.91% AUC and 16.89% accuracy compared with the previous state-of-the-art (SOTA) methods. The code is available at https://github.com/yuanzhang7/DSCENet.

DSCENet: Dynamic Screening and Clinical-Enhanced Multimodal Fusion for MPNs Subtype Classification

TL;DR

This work tackles the challenging problem of classifying myeloproliferative neoplasm subtypes by leveraging multimodal data from whole slide images and clinical information. The authors introduce DSCENet, which combines a Dynamic Screening Module for adaptive patch selection with a Clinical-enhanced Fusion Module that uses cross-modal attention to fuse image and clinical features. On a multicenter dataset of 383 cases, DSCENet achieves an ACC of 83.12% and an AUC around 96%, significantly outperforming state-of-the-art single-modality and fusion methods and demonstrating the value of dynamic patch screening and clinical guidance. Ablation studies corroborate the contributions of both modules, with clinical data providing substantial gains in AUC and patch screening improving feature representativeness, indicating strong potential for improving diagnostic accuracy and informing treatment planning in MPNs.

Abstract

The precise subtype classification of myeloproliferative neoplasms (MPNs) based on multimodal information, which assists clinicians in diagnosis and long-term treatment plans, is of great clinical significance. However, it remains a great challenging task due to the lack of diagnostic representativeness for local patches and the absence of diagnostic-relevant features from a single modality. In this paper, we propose a Dynamic Screening and Clinical-Enhanced Network (DSCENet) for the subtype classification of MPNs on the multimodal fusion of whole slide images (WSIs) and clinical information. (1) A dynamic screening module is proposed to flexibly adapt the feature learning of local patches, reducing the interference of irrelevant features and enhancing their diagnostic representativeness. (2) A clinical-enhanced fusion module is proposed to integrate clinical indicators to explore complementary features across modalities, providing comprehensive diagnostic information. Our approach has been validated on the real clinical data, achieving an increase of 7.91% AUC and 16.89% accuracy compared with the previous state-of-the-art (SOTA) methods. The code is available at https://github.com/yuanzhang7/DSCENet.
Paper Structure (17 sections, 4 equations, 4 figures, 2 tables)

This paper contains 17 sections, 4 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Dynamic Screening Module
  4. Clinical-enhanced Fusion Module
  5. Experiments
  6. Experimental Settings
  7. Dataset.
  8. Implementation Details.
  9. Evaluation Metric.
  10. Result and Analysis
  11. Quantitative Evaluation.
  12. Qualitative Evaluation.
  13. Ablation Study.
  14. Model analysis of confusion matrices.
  15. Conclusion
  16. ...and 2 more sections

Figures (4)

  • Figure 1: a) Challenge 1: Individual patches show high visual similarity across four subtypes, leading to confusing local features. b) Challenge 2: Single image modality only provides morphological information, lacking other diagnostic-related features.
  • Figure 2: The Framework of our DSCENet. The DS module aims to dynamically screen the local patches for better feature representation and clinical-enhanced fusion module explores the complementary diagnostic information.
  • Figure 3: ROC curves for different methods on PV, ET, PrePMF, and PMF.
  • Figure 4: Confusion matrices on the testing set, with true labels on the vertical axis and predictions on the horizontal axis. The deeper colors indicate higher accuracy.