Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FAST: A Dual-tier Few-Shot Learning Paradigm for Whole Slide Image Classification

Kexue Fu, Xiaoyuan Luo, Linhao Qu, Shuo Wang, Ying Xiong, Ilias Maglogiannis, Longxiang Gao, Manning Wang

TL;DR

This work proposes a novel and efficient dual-tier few-shot learning paradigm for WSI classification, named FAST, which significantly surpasses existing few-shot classification methods and approaches the accuracy of fully supervised methods with only 0.22$\%$ annotation costs.

Abstract

The expensive fine-grained annotation and data scarcity have become the primary obstacles for the widespread adoption of deep learning-based Whole Slide Images (WSI) classification algorithms in clinical practice. Unlike few-shot learning methods in natural images that can leverage the labels of each image, existing few-shot WSI classification methods only utilize a small number of fine-grained labels or weakly supervised slide labels for training in order to avoid expensive fine-grained annotation. They lack sufficient mining of available WSIs, severely limiting WSI classification performance. To address the above issues, we propose a novel and efficient dual-tier few-shot learning paradigm for WSI classification, named FAST. FAST consists of a dual-level annotation strategy and a dual-branch classification framework. Firstly, to avoid expensive fine-grained annotation, we collect a very small number of WSIs at the slide level, and annotate an extremely small number of patches. Then, to fully mining the available WSIs, we use all the patches and available patch labels to build a cache branch, which utilizes the labeled patches to learn the labels of unlabeled patches and through knowledge retrieval for patch classification. In addition to the cache branch, we also construct a prior branch that includes learnable prompt vectors, using the text encoder of visual-language models for patch classification. Finally, we integrate the results from both branches to achieve WSI classification. Extensive experiments on binary and multi-class datasets demonstrate that our proposed method significantly surpasses existing few-shot classification methods and approaches the accuracy of fully supervised methods with only 0.22$\%$ annotation costs. All codes and models will be publicly available on https://github.com/fukexue/FAST.

FAST: A Dual-tier Few-Shot Learning Paradigm for Whole Slide Image Classification

TL;DR

This work proposes a novel and efficient dual-tier few-shot learning paradigm for WSI classification, named FAST, which significantly surpasses existing few-shot classification methods and approaches the accuracy of fully supervised methods with only 0.22 annotation costs.

Abstract

The expensive fine-grained annotation and data scarcity have become the primary obstacles for the widespread adoption of deep learning-based Whole Slide Images (WSI) classification algorithms in clinical practice. Unlike few-shot learning methods in natural images that can leverage the labels of each image, existing few-shot WSI classification methods only utilize a small number of fine-grained labels or weakly supervised slide labels for training in order to avoid expensive fine-grained annotation. They lack sufficient mining of available WSIs, severely limiting WSI classification performance. To address the above issues, we propose a novel and efficient dual-tier few-shot learning paradigm for WSI classification, named FAST. FAST consists of a dual-level annotation strategy and a dual-branch classification framework. Firstly, to avoid expensive fine-grained annotation, we collect a very small number of WSIs at the slide level, and annotate an extremely small number of patches. Then, to fully mining the available WSIs, we use all the patches and available patch labels to build a cache branch, which utilizes the labeled patches to learn the labels of unlabeled patches and through knowledge retrieval for patch classification. In addition to the cache branch, we also construct a prior branch that includes learnable prompt vectors, using the text encoder of visual-language models for patch classification. Finally, we integrate the results from both branches to achieve WSI classification. Extensive experiments on binary and multi-class datasets demonstrate that our proposed method significantly surpasses existing few-shot classification methods and approaches the accuracy of fully supervised methods with only 0.22 annotation costs. All codes and models will be publicly available on https://github.com/fukexue/FAST.
Paper Structure (28 sections, 10 equations, 6 figures, 4 tables)

This paper contains 28 sections, 10 equations, 6 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. WSI Classification
  4. Few-shot Learning for WSI Classification
  5. Vision-Language Model Adaptation
  6. Method
  7. Annotation Strategy and Problem Formulation
  8. Classification Framework
  9. 1) Image Cache Branch
  10. 2) CLIP Prior Knowledge Branch
  11. Experiments
  12. Datasets and Few-Shot Scenario Simulation
  13. Comparing Methods and Evaluation Metrics
  14. Experimental Results
  15. CAMELYON16
  16. ...and 13 more sections

Figures (6)

  • Figure 1: Different few-shot learning paradigms for WSI classification. (a) The instance few-shot method divides all WSIs into a series of patches, then selects a few samples at the patch level and annotates them at the patch level. The red box represents positive samples, and the blue box represents negative samples. (b) The bag few-shot method directly selects a few WSIs at the slide level and annotates them weakly at the slide level. (c) Our method first selects a few WSIs at the slide level, then annotates a few patches for each selected WSI. Compared to (a) and (b), our method significantly reduces annotation costs while providing patch-level supervision information.
  • Figure 2: The structure of the FAST classification framework.
  • Figure 3: Results of FAST on CAMELYON16 dataset under different annotation ratio.
  • Figure 4: Comparison of cache branch and prior branch in FAST.
  • Figure 5: Results of FAST on TCGA-RENAL dataset under different annotation ratio.
  • ...and 1 more figures