Look a Group at Once: Multi-Slide Modeling for Survival Prediction

TL;DR

GroupMIL is introduced, a novel framework inspired by the clinical practice of collective analysis, which models multiple slides as a single sample and organizes groups of patches and slides sequentially to capture cross-slide prognostic features and develops a dual-head predictor that delivers comprehensive survival risk and probability assessments for each patient.

Abstract

Survival prediction is a critical task in pathology. In clinical practice, pathologists often examine multiple cases, leveraging a broader spectrum of cancer phenotypes to enhance pathological assessment. Despite significant advancements in deep learning, current solutions typically model each slide as a sample, struggling to effectively capture comparable and slide-agnostic pathological features. In this paper, we introduce GroupMIL, a novel framework inspired by the clinical practice of collective analysis, which models multiple slides as a single sample and organizes groups of patches and slides sequentially to capture cross-slide prognostic features. We also present GPAMamba, a model designed to facilitate intra- and inter-slide feature interactions, effectively capturing local micro-environmental characteristics within slide-level graphs while uncovering essential prognostic patterns across an extended patch sequence within the group framework. Furthermore, we develop a dual-head predictor that delivers comprehensive survival risk and probability assessments for each patient. Extensive empirical evaluations demonstrate that our model significantly outperforms state-of-the-art approaches across five datasets from The Cancer Genome Atlas.

Figures (8)

• Figure 1: (a) illustrates the process by which pathologists reference other slides and compare phenotypes to enhance their assessments. (b) depicts the conventional procedure of processing WSIs individually. (c) presents our proposed group modeling.
• Figure 2: In our framework, each group of slides is segmented into patches, encoded, and represented as independent graphs before being input into GPAMamba. Within GPAMamba, node features are aggregated and sequentially scanned (details in Sec. \ref{['subsec:PAMamba']}). The resulting graphs are then pooled to obtain slide representations, which are also arranged sequentially before being analyzed by a Mamba module. Finally, these representations are evaluated using the dual-head predictor, comprising $\mathcal{H}_{\text{risk}}$ and $\mathcal{H}_{\text{prob}}$ (details in Sec. \ref{['subsec:heads']}).
• Figure 3: The standard pipeline for survival prediction.
• Figure 4: Overview of GPAMamba. GC layers convolve the graphs $\{s_1,...,s_b\}$, while PAMamba scans the sequence $g$. The red lines indicate the position scanning branch, and the blue lines represent the attention scanning branch.
• Figure 5: The figure shows a comparison of FLOPs and GPU memory usage between Transformer-based and Mamba-based modules in our framework.