Joint Modelling Histology and Molecular Markers for Cancer Classification

TL;DR

This work tackles glioma classification under the WHO 2021 molecular pathology framework by jointly predicting histology features and molecular markers from whole-slide images. It introduces the M3C2 framework, combining a multi-scale disentangling module, an attention-based hierarchical multi-task MIL, a co-occurrence label graph for molecular markers, and a cross-modal interaction pipeline with dynamic gradient modulation and confidence-based losses. The approach demonstrates superior performance and generalizability on internal and external glioma datasets, with ablations confirming the value of each component and multi-scale fusion. By explicitly modeling histology–molecular marker interactions and leveraging multi-scale information, M3C2 holds promise for more accurate, interpretable, and cost-effective cancer diagnosis and precision oncology workflows.

Abstract

Cancers are characterized by remarkable heterogeneity and diverse prognosis. Accurate cancer classification is essential for patient stratification and clinical decision-making. Although digital pathology has been advancing cancer diagnosis and prognosis, the paradigm in cancer pathology has shifted from purely relying on histology features to incorporating molecular markers. There is an urgent need for digital pathology methods to meet the needs of the new paradigm. We introduce a novel digital pathology approach to jointly predict molecular markers and histology features and model their interactions for cancer classification. Firstly, to mitigate the challenge of cross-magnification information propagation, we propose a multi-scale disentangling module, enabling the extraction of multi-scale features from high-magnification (cellular-level) to low-magnification (tissue-level) whole slide images. Further, based on the multi-scale features, we propose an attention-based hierarchical multi-task multi-instance learning framework to simultaneously predict histology and molecular markers. Moreover, we propose a co-occurrence probability-based label correlation graph network to model the co-occurrence of molecular markers. Lastly, we design a cross-modal interaction module with the dynamic confidence constrain loss and a cross-modal gradient modulation strategy, to model the interactions of histology and molecular markers. Our experiments demonstrate that our method outperforms other state-of-the-art methods in classifying glioma, histology features and molecular markers. Our method promises to promote precise oncology with the potential to advance biomedical research and clinical applications. The code is available at https://github.com/LHY1007/M3C2

Figures (7)

• Figure 1: Framework of our M3C2 method, including (A) multi-scale disentangling module, (B) molecular prediction module, (C) cross-module interaction module and (D) histology prediction module. Note that IM, SM, IH and SH denote independent molecular features, shared molecular features, independent histology features and shared histology features, respectively.
• Figure 2: Detailed structure of the proposed molecular prediction module (above) and the histology prediction module (below).
• Figure 3: Detailed structure of the proposed CPLC-Graph network and the LC loss.
• Figure 4: Illustration of the CMG-Modu learning strategy
• Figure 5: ROCs of our model, comparison and ablation models for predicting IDH, 1p/19q, CDKN, NMP and Glioma.