Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Joint multi-task learning improves weakly-supervised biomarker prediction in computational pathology

Omar S. M. El Nahhas, Georg Wölflein, Marta Ligero, Tim Lenz, Marko van Treeck, Firas Khader, Daniel Truhn, Jakob Nikolas Kather

TL;DR

This work developed a weakly-supervised joint multi-task Transformer architecture which has been trained and evaluated on four public patient cohorts for the prediction of two key predictive biomarkers, microsatellite instability (MSI) and homologous recombination deficiency (HRD), trained with auxiliary regression tasks related to the tumor microenvironment.

Abstract

Deep Learning (DL) can predict biomarkers directly from digitized cancer histology in a weakly-supervised setting. Recently, the prediction of continuous biomarkers through regression-based DL has seen an increasing interest. Nonetheless, clinical decision making often requires a categorical outcome. Consequently, we developed a weakly-supervised joint multi-task Transformer architecture which has been trained and evaluated on four public patient cohorts for the prediction of two key predictive biomarkers, microsatellite instability (MSI) and homologous recombination deficiency (HRD), trained with auxiliary regression tasks related to the tumor microenvironment. Moreover, we perform a comprehensive benchmark of 16 approaches of task balancing for weakly-supervised joint multi-task learning in computational pathology. Using our novel approach, we improve over the state-of-the-art area under the receiver operating characteristic by +7.7% and +4.1%, as well as yielding better clustering of latent embeddings by +8% and +5% for the prediction of MSI and HRD in external cohorts, respectively.

Joint multi-task learning improves weakly-supervised biomarker prediction in computational pathology

TL;DR

This work developed a weakly-supervised joint multi-task Transformer architecture which has been trained and evaluated on four public patient cohorts for the prediction of two key predictive biomarkers, microsatellite instability (MSI) and homologous recombination deficiency (HRD), trained with auxiliary regression tasks related to the tumor microenvironment.

Abstract

Deep Learning (DL) can predict biomarkers directly from digitized cancer histology in a weakly-supervised setting. Recently, the prediction of continuous biomarkers through regression-based DL has seen an increasing interest. Nonetheless, clinical decision making often requires a categorical outcome. Consequently, we developed a weakly-supervised joint multi-task Transformer architecture which has been trained and evaluated on four public patient cohorts for the prediction of two key predictive biomarkers, microsatellite instability (MSI) and homologous recombination deficiency (HRD), trained with auxiliary regression tasks related to the tumor microenvironment. Moreover, we perform a comprehensive benchmark of 16 approaches of task balancing for weakly-supervised joint multi-task learning in computational pathology. Using our novel approach, we improve over the state-of-the-art area under the receiver operating characteristic by +7.7% and +4.1%, as well as yielding better clustering of latent embeddings by +8% and +5% for the prediction of MSI and HRD in external cohorts, respectively.
Paper Structure (16 sections, 3 figures, 3 tables)

This paper contains 16 sections, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Method
  4. Feature extraction
  5. Architecture
  6. Training
  7. Multi-task balancing
  8. Experiments and Results
  9. Data
  10. Joint multi-task learning improves classification predictions
  11. Joint multi-task learning improves generalizability
  12. Joint multi-task learning improves latent-embedding clustering
  13. Conclusion
  14. Acknowledgements.
  15. Competing interests.
  16. ...and 1 more sections

Figures (3)

  • Figure 1: Model overview. We tessellate into patches, extract CTransPath features Wang2022-wg, linearly project them, and feed them into a Transformer encoder. A learnable classification and regression token are added to the input of the Transformer decoder, after which the output is fed to a classification and regression head, performing weakly-supervised joint multi-task learning with weighting- and gradient-based task balancing.
  • Figure 1: The relationship between TME signatures, and a,b MSI and c,d HRD. TME signatures with non-random relationships with MSI and HRD were used for subsequent experiments.
  • Figure 2: Visualization of the classification and joint-learned embeddings for MSI of the external cohort (n=105) using t-SNE