Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TAKT: Target-Aware Knowledge Transfer for Whole Slide Image Classification

Conghao Xiong, Yi Lin, Hao Chen, Hao Zheng, Dong Wei, Yefeng Zheng, Joseph J. Y. Sung, Irwin King

TL;DR

WSI classification is hampered by limited labeled data and cross-domain shifts. The authors propose Target-Aware Knowledge Transfer (TAKT), which trains a teacher on Target-Aware Data Augmentation (TADA) and aligns source-target features with a Target-Aware Feature Alignment (TAFA) module based on Optimal Transport. The teacher is trained with unlabelled target data and guides a student on the target domain through a transfer loss that leverages a Multi-Head Attention adaptation, with the OT-based distance regularizing cross-domain features. Experiments on Camelyon16, TCGA-RCC, and TCGA-NSCLC show that TAKT outperforms baselines and achieves state-of-the-art performance among knowledge-transfer methods for cross-dataset WSI classification.

Abstract

Transferring knowledge from a source domain to a target domain can be crucial for whole slide image classification, since the number of samples in a dataset is often limited due to high annotation costs. However, domain shift and task discrepancy between datasets can hinder effective knowledge transfer. In this paper, we propose a Target-Aware Knowledge Transfer framework, employing a teacher-student paradigm. Our framework enables the teacher model to learn common knowledge from the source and target domains by actively incorporating unlabelled target images into the training of the teacher model. The teacher bag features are subsequently adapted to supervise the training of the student model on the target domain. Despite incorporating the target features during training, the teacher model tends to overlook them under the inherent domain shift and task discrepancy. To alleviate this, we introduce a target-aware feature alignment module to establish a transferable latent relationship between the source and target features by solving the optimal transport problem. Experimental results show that models employing knowledge transfer outperform those trained from scratch, and our method achieves state-of-the-art performance among other knowledge transfer methods on various datasets, including TCGA-RCC, TCGA-NSCLC, and Camelyon16.

TAKT: Target-Aware Knowledge Transfer for Whole Slide Image Classification

TL;DR

WSI classification is hampered by limited labeled data and cross-domain shifts. The authors propose Target-Aware Knowledge Transfer (TAKT), which trains a teacher on Target-Aware Data Augmentation (TADA) and aligns source-target features with a Target-Aware Feature Alignment (TAFA) module based on Optimal Transport. The teacher is trained with unlabelled target data and guides a student on the target domain through a transfer loss that leverages a Multi-Head Attention adaptation, with the OT-based distance regularizing cross-domain features. Experiments on Camelyon16, TCGA-RCC, and TCGA-NSCLC show that TAKT outperforms baselines and achieves state-of-the-art performance among knowledge-transfer methods for cross-dataset WSI classification.

Abstract

Transferring knowledge from a source domain to a target domain can be crucial for whole slide image classification, since the number of samples in a dataset is often limited due to high annotation costs. However, domain shift and task discrepancy between datasets can hinder effective knowledge transfer. In this paper, we propose a Target-Aware Knowledge Transfer framework, employing a teacher-student paradigm. Our framework enables the teacher model to learn common knowledge from the source and target domains by actively incorporating unlabelled target images into the training of the teacher model. The teacher bag features are subsequently adapted to supervise the training of the student model on the target domain. Despite incorporating the target features during training, the teacher model tends to overlook them under the inherent domain shift and task discrepancy. To alleviate this, we introduce a target-aware feature alignment module to establish a transferable latent relationship between the source and target features by solving the optimal transport problem. Experimental results show that models employing knowledge transfer outperform those trained from scratch, and our method achieves state-of-the-art performance among other knowledge transfer methods on various datasets, including TCGA-RCC, TCGA-NSCLC, and Camelyon16.
Paper Structure (23 sections, 3 equations, 1 figure, 3 tables)

This paper contains 23 sections, 3 equations, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Overview of the Target-Aware Knowledge Transfer
  4. Target-Aware Data Augmentation
  5. Overview of the TADA Method.
  6. Step 1: Clustering of the Target Dataset.
  7. Step 2: Cross-Dataset Mix Operation.
  8. Target-Aware Feature Alignment Module
  9. Experiments and Results
  10. Dataset Descriptions
  11. TCGA-RCC (RCC).
  12. TCGA-NSCLC (NSCLC).
  13. Implementation Details
  14. Evaluation Metrics.
  15. Training Settings.
  16. ...and 8 more sections

Figures (1)

  • Figure 1: Illustrations of (a) our target-aware knowledge transfer framework, (b) our target-aware data augmentation method and (c) our target-aware feature alignment module and a sample optimal transport flow. $\mathcal{M}_{t}(\cdot)$ is the teacher model and $\mathcal{M}_{s}(\cdot)$ is the student model. The light areas in (c) indicate regions with higher values.