Triage of 3D pathology data via 2.5D multiple-instance learning to guide pathologist assessments

TL;DR

This work tackles the under-sampling challenge in histopathology by introducing CARP3D, a 2.5D multiple-instance learning framework that triages high-risk slices within large 3D pathology datasets. It combines intra-slice attention-based MIL on patch features with inter-slice pooling to incorporate depth context, producing a context-aware risk score for each slice at the axial sampling pitch $1\,\mu\mathrm{m}$. In prostate cancer OTLS datasets, CARP3D outperforms 2D baselines and demonstrates interpretability through attention heatmaps and PCA, achieving a high slice-triage performance (e.g., $\text{AUC}$ improvements from $\approx81\%$ to $\approx90\%$ with weighted 2.5D pooling). The approach offers a practical pathway toward clinical adoption by guiding pathologists to the most informative slices while preserving human expertise for final diagnosis, with future validation across organs and larger cohorts.

Abstract

Accurate patient diagnoses based on human tissue biopsies are hindered by current clinical practice, where pathologists assess only a limited number of thin 2D tissue slices sectioned from 3D volumetric tissue. Recent advances in non-destructive 3D pathology, such as open-top light-sheet microscopy, enable comprehensive imaging of spatially heterogeneous tissue morphologies, offering the feasibility to improve diagnostic determinations. A potential early route towards clinical adoption for 3D pathology is to rely on pathologists for final diagnosis based on viewing familiar 2D H&E-like image sections from the 3D datasets. However, manual examination of the massive 3D pathology datasets is infeasible. To address this, we present CARP3D, a deep learning triage approach that automatically identifies the highest-risk 2D slices within 3D volumetric biopsy, enabling time-efficient review by pathologists. For a given slice in the biopsy, we estimate its risk by performing attention-based aggregation of 2D patches within each slice, followed by pooling of the neighboring slices to compute a context-aware 2.5D risk score. For prostate cancer risk stratification, CARP3D achieves an area under the curve (AUC) of 90.4% for triaging slices, outperforming methods relying on independent analysis of 2D sections (AUC=81.3%). These results suggest that integrating additional depth context enhances the model's discriminative capabilities. In conclusion, CARP3D has the potential to improve pathologist diagnosis via accurate triage of high-risk slices within large-volume 3D pathology datasets.

Figures (5)

• Figure 1: Workflow with deep-learning-based triage framework for 3D pathology. Prostate biopsies are comprehensively imaged in 3D with open-top light-sheet (OTLS) microscopy. A deep-learning-based triage method evaluates all 2D slices within 3D pathology datasets and identifies the highest-risk 2D slices for time-efficient pathologist review.
• Figure 2: $\textsc{CARP3D}$ architecture. a) Patches for a 2D slice of interest (SOI) and its neighboring slices are encoded with pretrained ResNet50 and CTransPath. An intra-slice attention module aggregates patch-level features within each slice into slice-level features. Neighboring slice-level features are aggregated through an inter-slice pooling module to produce a context-aware SOI feature for subsequent risk prediction, formulated as a classification task into high- vs. low-risk categories here. b) During training, slices are selected within each 3D sample of the training set, from which the $\textsc{CARP3D}$ model learns to predict the ground truth labels provided by pathologists. c) Model deployment on 3D pathology data for slice-by-slice risk assessment. The highest-risk slices are selected for pathologist review.
• Figure 3: Baseline architectures for inter-slice pooling. a) Naive pooling. b) Average pooling. c) RNN-based pooling.
• Figure 4: Visualization of SOI features and interpretable attention heatmaps. a) PCA of context-aware SOI features. b) and c) show examples of attention heatmaps with corresponding false-colored images. The scale bar is 100 $\mu m$. Color bar indicates attention scores.
• Figure 5: $\textsc{CARP3D}$ triage on an example 3D pathology dataset. a) An example 3D pathology dataset. b) Per-slice risk profile, predicted by $\textsc{CARP3D}$, for higher-grade prostate cancer. c) Images at arrow positions are reviewed by a board-certified pathologist, showing that human evaluation on select slices broadly aligns with the risk profile. The scale bar is 100 $\mu m$.