Transfer Learning from One Cancer to Another via Deep Learning Domain Adaptation

TL;DR

This work tackles domain shift in cancer histopathology by applying a domain-adversarial neural network (DANN) to transfer lesion-classification knowledge across organ sites. Using a ResNet-50 backbone, the model learns domain-invariant features by training on labeled breast and colon adenocarcinomas while adapting to unlabeled lung (and other) domains, with stain normalization explored as a domain-gap mitigator. Key findings show strong target-domain performance for lung (≈95.56%) and colon (≈78.48%) when kidney data are excluded, while breast and kidney targets are harder to transfer; stain normalization can improve some targets but may hurt others. Explainability via Integrated Gradients reveals attribution to densely packed nuclei, aligning with clinically relevant cancer features and enhancing trust in cross-domain predictions. These results underscore the potential of domain-adaptive approaches in multicenter oncologic imaging and highlight the importance of imaging modality compatibility and explainability for clinical adoption.

Abstract

Supervised deep learning models often achieve excellent performance within their training distribution but struggle to generalize beyond it. In cancer histopathology, for example, a convolutional neural network (CNN) may classify cancer severity accurately for cancer types represented in its training data, yet fail on related but unseen types. Although adenocarcinomas from different organs share morphological features that might support limited cross-domain generalization, addressing domain shift directly is necessary for robust performance. Domain adaptation offers a way to transfer knowledge from labeled data in one cancer type to unlabeled data in another, helping mitigate the scarcity of annotated medical images. This work evaluates cross-domain classification performance among lung, colon, breast, and kidney adenocarcinomas. A ResNet50 trained on any single adenocarcinoma achieves over 98% accuracy on its own domain but shows minimal generalization to others. Ensembling multiple supervised models does not resolve this limitation. In contrast, converting the ResNet50 into a domain adversarial neural network (DANN) substantially improves performance on unlabeled target domains. A DANN trained on labeled breast and colon data and adapted to unlabeled lung data reaches 95.56% accuracy. We also examine the impact of stain normalization on domain adaptation. Its effects vary by target domain: for lung, accuracy drops from 95.56% to 66.60%, while for breast and colon targets, stain normalization boosts accuracy from 49.22% to 81.29% and from 78.48% to 83.36%, respectively. Finally, using Integrated Gradients reveals that DANNs consistently attribute importance to biologically meaningful regions such as densely packed nuclei, indicating that the model learns clinically relevant features and can apply them to unlabeled cancer types.

Figures (6)

• Figure 1: Example of a breast histopathology image. This example is labeled as malignant ($y=1$).
• Figure 2: DANN architecture with ResNet-50 feature extractor used in our domain adaptation experiments.
• Figure 3: Baseline domain generalization results on the adenocarcinoma dataset
• Figure 4: Ensemble model results: titled domain represents the domain that was left out in the leave-one-out ensemble approach and the domain present within the test dataset.
• Figure 5: Example of the Integrated Gradients attribution map for the breast target domain DANN for a breast histopathology image from the test set depicted in Fig. \ref{['figExampleIntegratedGradientsOriginal']}. The attribution map is colored such that white indicates a positive attribution of a given pixel to the final prediction, while black indicates a negative attribution.