Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SimMIL: A Universal Weakly Supervised Pre-Training Framework for Multi-Instance Learning in Whole Slide Pathology Images

Yicheng Song, Tiancheng Lin, Die Peng, Su Yang, Yi Xu

TL;DR

SimMIL addresses the gap in MIL for WSIs by learning instance representations through weakly supervised pre-training that propagates bag-level labels to individual patches. It combines strong data augmentation, a non-linear prediction head, and task-specific losses to train a robust feature extractor, improving downstream benign-malignant classification, cancer subtyping, and survival prediction beyond ImageNet and SSL baselines. The framework shows compatibility with existing SSL models and scalability when merging datasets, supported by comprehensive ablations and visualizations. Overall, SimMIL demonstrates that MIL-oriented representation learning can be effectively achieved via simple, weakly supervised pre-training, enabling stronger bag-level MIL performance and serving as a strong baseline for future MIL studies in pathology.

Abstract

Various multi-instance learning (MIL) based approaches have been developed and successfully applied to whole-slide pathological images (WSI). Existing MIL methods emphasize the importance of feature aggregators, but largely neglect the instance-level representation learning. They assume that the availability of a pre-trained feature extractor can be directly utilized or fine-tuned, which is not always the case. This paper proposes to pre-train feature extractor for MIL via a weakly-supervised scheme, i.e., propagating the weak bag-level labels to the corresponding instances for supervised learning. To learn effective features for MIL, we further delve into several key components, including strong data augmentation, a non-linear prediction head and the robust loss function. We conduct experiments on common large-scale WSI datasets and find it achieves better performance than other pre-training schemes (e.g., ImageNet pre-training and self-supervised learning) in different downstream tasks. We further show the compatibility and scalability of the proposed scheme by deploying it in fine-tuning the pathological-specific models and pre-training on merged multiple datasets. To our knowledge, this is the first work focusing on the representation learning for MIL.

SimMIL: A Universal Weakly Supervised Pre-Training Framework for Multi-Instance Learning in Whole Slide Pathology Images

TL;DR

SimMIL addresses the gap in MIL for WSIs by learning instance representations through weakly supervised pre-training that propagates bag-level labels to individual patches. It combines strong data augmentation, a non-linear prediction head, and task-specific losses to train a robust feature extractor, improving downstream benign-malignant classification, cancer subtyping, and survival prediction beyond ImageNet and SSL baselines. The framework shows compatibility with existing SSL models and scalability when merging datasets, supported by comprehensive ablations and visualizations. Overall, SimMIL demonstrates that MIL-oriented representation learning can be effectively achieved via simple, weakly supervised pre-training, enabling stronger bag-level MIL performance and serving as a strong baseline for future MIL studies in pathology.

Abstract

Various multi-instance learning (MIL) based approaches have been developed and successfully applied to whole-slide pathological images (WSI). Existing MIL methods emphasize the importance of feature aggregators, but largely neglect the instance-level representation learning. They assume that the availability of a pre-trained feature extractor can be directly utilized or fine-tuned, which is not always the case. This paper proposes to pre-train feature extractor for MIL via a weakly-supervised scheme, i.e., propagating the weak bag-level labels to the corresponding instances for supervised learning. To learn effective features for MIL, we further delve into several key components, including strong data augmentation, a non-linear prediction head and the robust loss function. We conduct experiments on common large-scale WSI datasets and find it achieves better performance than other pre-training schemes (e.g., ImageNet pre-training and self-supervised learning) in different downstream tasks. We further show the compatibility and scalability of the proposed scheme by deploying it in fine-tuning the pathological-specific models and pre-training on merged multiple datasets. To our knowledge, this is the first work focusing on the representation learning for MIL.
Paper Structure (47 sections, 5 equations, 6 figures, 9 tables)

This paper contains 47 sections, 5 equations, 6 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Multi-instance Learning for WSI Analysis
  4. Visual Representation Learning
  5. Pretext Tasks
  6. Methods
  7. Background
  8. SimMIL for WSI analysis
  9. Evaluation protocols
  10. Discussion and analysis
  11. Pre-pretraining scheme
  12. Scaling laws
  13. Intuitive understanding
  14. Experiments and results
  15. Dataset
  16. ...and 32 more sections

Figures (6)

  • Figure 1: Typical training schemes.
  • Figure 2: Overview of SimMIL: the pre-training process and downstream MIL tasks. The t-SNE in the bottom left is the visualization result on NCTCRC since the instance-level labels are available.
  • Figure 3: Reuslts (%) of bag-level classification with feature extractor released and fine-tuned CTransPath on Camelyon16 and TCGA-NSCLC, HIPT on TCGA-NSCLC and TCGA-BRCA. FT-Epoch1 denotes fine-tuning for 1 epoch and analogously for FT-Epoch5 and FT-Epoch10. Acc and AUC are reported by solid-colored and striped bars. Legend is the same for 4 subfigures.
  • Figure 4: The t-SNE visualization of instance level features. Three classes, normal, tumor and nontumor, denote positive instances in positive bags, negative instances in negative and positive bags, respectively. Features are generated by different pre-training methods on Camelyon16. Legend is the same for 6 subfigures.
  • Figure 5: Attention map of CLAM-SB aggregator using features from different pre-training paradigms. The WSI is from Camelyon16.
  • ...and 1 more figures