Cross-Cancer Knowledge Transfer in WSI-based Prognosis Prediction

TL;DR

This work recasts WSI-based prognosis prediction from a cancer-specific paradigm to cross-cancer knowledge transfer. It introduces UNI2-h-DSS, a 26-cancer dataset, and CROPKT to study transferability, including a routing-based baseline ROUPKT that leverages off-the-shelf models. The study finds both negative and positive transfer across cancers, identifies intra- and inter-task factors guiding transfer, and demonstrates a 3.1% average improvement using ROUPKT. These findings lay the groundwork for scalable, cross-cancer prognosis in WSIs and highlight routing as a powerful mechanism for integrating diverse prognostic knowledge.

Abstract

Whole-Slide Image (WSI) is an important tool for estimating cancer prognosis. Current studies generally follow a conventional cancer-specific paradigm in which each cancer corresponds to a single model. However, this paradigm naturally struggles to scale to rare tumors and cannot leverage knowledge from other cancers. While multi-task learning frameworks have been explored recently, they often place high demands on computational resources and require extensive training on ultra-large, multi-cancer WSI datasets. To this end, this paper shifts the paradigm to knowledge transfer and presents the first preliminary yet systematic study on cross-cancer prognosis knowledge transfer in WSIs, called CROPKT. It comprises three major parts. (1) We curate a large dataset (UNI2-h-DSS) with 26 cancers and use it to measure the transferability of WSI-based prognostic knowledge across different cancers (including rare tumors). (2) Beyond a simple evaluation merely for benchmarking, we design a range of experiments to gain deeper insights into the underlying mechanism behind transferability. (3) We further show the utility of cross-cancer knowledge transfer, by proposing a routing-based baseline approach (ROUPKT) that could often efficiently utilize the knowledge transferred from off-the-shelf models of other cancers. CROPKT could serve as an inception that lays the foundation for this nascent paradigm, i.e., WSI-based prognosis prediction with cross-cancer knowledge transfer. Our source code is available at https://github.com/liupei101/CROPKT.

Figures (11)

• Figure 1: Dataset curation and statistics. C and R refer to common and rare cancer diseases.
• Figure 2: Performance of cross-cancer knowledge transfer ($\mathcal{S}\to\mathcal{T}$). Cancer-specific models are first trained on their respective datasets and are then transferred to other cancers for survival prediction. $\mathcal{M}_{\mathcal{S}}$ is evaluated on the five-fold test data from $\mathcal{T}$ to report transfer performance (C-Index).
• Figure 3: Visualization of tissue annotations and attention heatmaps to showcase representative knowledge that $\mathcal{M}_{\mathcal{S}\to\mathcal{T}}$ offers: (a) overlapping and useful, (b) dissimilar and useless, and (c) dissimilar yet useful, where (a) and (c) help to positive transfer. Refer to Appendix \ref{['sec:apx_c_1']} for more results.
• Figure 4: Univariate analysis and visualization for intra-task and inter-task factors.
• Figure 5: A schematic diagram of our routing-based baseline approach ($\textsc{ROUPKT}$) to utilizing the knowledge transferred from other cancers.