Enabling Progressive Whole-slide Image Analysis with Multi-scale Pyramidal Network

TL;DR

This work tackles the rigidity and computational cost of existing multi-scale MIL for WSIs in computational pathology. It introduces MSPN, a plug-and-play module that derives coarse multi-scale context from high-magnification features via grid-based remapping and residual CGNs, enabling progressive multi-scale analysis without multiple inputs. Across four clinically relevant tasks and three foundation-model features, MSPN delivers consistent improvements over traditional multi-scale schemes while maintaining lightweight efficiency, and it remains effective when integrated with pre-trained MIL systems. The approach enhances interpretability through heatmap-guided attention and offers a practical pathway to scalable, multi-scale CPath analyses.

Abstract

Multiple-instance Learning (MIL) is commonly used to undertake computational pathology (CPath) tasks, and the use of multi-scale patches allows diverse features across scales to be learned. Previous studies using multi-scale features in clinical applications rely on multiple inputs across magnifications with late feature fusion, which does not retain the link between features across scales while the inputs are dependent on arbitrary, manufacturer-defined magnifications, being inflexible and computationally expensive. In this paper, we propose the Multi-scale Pyramidal Network (MSPN), which is plug-and-play over attention-based MIL that introduces progressive multi-scale analysis on WSI. Our MSPN consists of (1) grid-based remapping that uses high magnification features to derive coarse features and (2) the coarse guidance network (CGN) that learns coarse contexts. We benchmark MSPN as an add-on module to 4 attention-based frameworks using 4 clinically relevant tasks across 3 types of foundation model, as well as the pre-trained MIL framework. We show that MSPN consistently improves MIL across the compared configurations and tasks, while being lightweight and easy-to-use.

Figures (7)

• Figure 1: Different schemes for multi-scale MIL. (a), (b) Either concatenation or cross-scale attention rely on multiple inputs deng2024cross-scale, and magnifications are fixed in preprocessing. (c) The proposed MSPN consists of multiple CGNs that derive coarser guidances with adjustable scales from high magnification features.
• Figure 2: Overview of the Multi-scale Pyramidal Network (MSPN) in the universal MIL pipeline. (1) The MSPN consists of $k$ coarse guidance networks (CGN) with residual connections and each CGN serves to generate guidances at different coarser scales. (2) The coarse grids for guidances are determined by the selected field-of-view (FOV) using coordinates in the original high magnification, retaining the aspect ratio. No extra pre-processing is needed since grid-based remapping is used for mapping high magnification features into the proper coarse grids. (3) Features in each grid are aggregated to form a coarse feature map for the CGN, and a sigmoid gate is learned to score the high magnification patches using the coarse feature map.
• Figure 3: Runtime comparison under features in different dimensions. MSPN is more efficient when feature dimension scales up.
• Figure 4: Examples of heatmap visualisations on prognosis prediction. The coarse guidances from CGNs with FOV of 3072, 2048, and 1536 are separately shown, indicating a pattern of progressive focus on the important areas. The suggested area of coarse guidance aligns with the heatmaps from high magnification features.
• Figure 5: Runtime comparison under features in different dimensions. MSPN is more efficient when feature dimension scales up.