Pathological Prior-Guided Multiple Instance Learning For Mitigating Catastrophic Forgetting in Breast Cancer Whole Slide Image Classification

TL;DR

The paper tackles catastrophic forgetting in breast cancer WSI classification under continual learning by introducing PaGMIL, which integrates micro- and macro-pathology priors into a MIL framework. A Patch Selector leverages spatial cancer clustering and patch coordinates to curate diverse, representative patches, while a Prompt Guide uses WSI thumbnails to generate task-specific prompts and assign per-task classification heads via intra- and inter-task losses. Across four public datasets (CAMELYON16, BRACS, TCGA-BRCA, BACH), PaGMIL demonstrates a superior balance between current-task performance and retention of previous knowledge, outperforming baselines and recent continual-learning methods. The approach enhances robustness to cross-institution variability and offers a practical path toward scalable, continual WSI analysis in histopathology.

Abstract

In histopathology, intelligent diagnosis of Whole Slide Images (WSIs) is essential for automating and objectifying diagnoses, reducing the workload of pathologists. However, diagnostic models often face the challenge of forgetting previously learned data during incremental training on datasets from different sources. To address this issue, we propose a new framework PaGMIL to mitigate catastrophic forgetting in breast cancer WSI classification. Our framework introduces two key components into the common MIL model architecture. First, it leverages microscopic pathological prior to select more accurate and diverse representative patches for MIL. Secondly, it trains separate classification heads for each task and uses macroscopic pathological prior knowledge, treating the thumbnail as a prompt guide (PG) to select the appropriate classification head. We evaluate the continual learning performance of PaGMIL across several public breast cancer datasets. PaGMIL achieves a better balance between the performance of the current task and the retention of previous tasks, outperforming other continual learning methods. Our code will be open-sourced upon acceptance.

Figures (3)

• Figure 1: Illustration of dynamic input of WSI data. The actual model update involves different medical institutions sending in some data for training. Significant differences in data from different batches arise due to variations in staining methods or equipment used across different medical institutions.
• Figure 2: Overview of our proposed PaGMIL method. Based on the MIL pipeline, we add two new modules. The Patch Selector module uses microscopic pathology prior to select more accurate representative patches, and the PG module uses macroscopic pathology priors to select the correct classification head.
• Figure 3: The image is from CAMELYON16, which is WSI with cancer. Red indicates high scores given by the scoring network, while blue indicates low scores. The models are trained sequentially from left to right in the order of datasets. As training progresses, the baseline method shows an opposite distribution of red and blue, changing the representative patches from cancerous to normal tissue. In contrast, our method maintains a similar distribution of red and blue.