Digital Volumetric Biopsy Cores Improve Gleason Grading of Prostate Cancer Using Deep Learning

TL;DR

A novel digital pathology data source called a “volumetric core” is proposed, obtained via the extraction and co-alignment of serially sectioned tissue sections using a novel morphology-preserving alignment framework, obtained via the extraction and co-alignment of serially sectioned tissue sections.

Abstract

Prostate cancer (PCa) was the most frequently diagnosed cancer among American men in 2023. The histological grading of biopsies is essential for diagnosis, and various deep learning-based solutions have been developed to assist with this task. Existing deep learning frameworks are typically applied to individual 2D cross-sections sliced from 3D biopsy tissue specimens. This process impedes the analysis of complex tissue structures such as glands, which can vary depending on the tissue slice examined. We propose a novel digital pathology data source called a "volumetric core," obtained via the extraction and co-alignment of serially sectioned tissue sections using a novel morphology-preserving alignment framework. We trained an attention-based multiple-instance learning (ABMIL) framework on deep features extracted from volumetric patches to automatically classify the Gleason Grade Group (GGG). To handle volumetric patches, we used a modified video transformer with a deep feature extractor pretrained using self-supervised learning. We ran our morphology-preserving alignment framework to construct 10,210 volumetric cores, leaving out 30% for pretraining. The rest of the dataset was used to train ABMIL, which resulted in a 0.958 macro-average AUC, 0.671 F1 score, 0.661 precision, and 0.695 recall averaged across all five GGG significantly outperforming the 2D baselines.

Figures (5)

• Figure 1: VCore computational workflow. (A. Morphology-preserving tissue alignment framework consists of 3 main steps: individual tissue ribbon extraction, serial rigid registration, and high-resolution non-rigid registration based on the boundary.) (B. VCore first separates the core tissue from the background and then splits it into the volumetric patches without overlap, removing patches containing less than 60% of tissue.) (C. Self-supervised pretraining framework based on DINO with TimeSformer backbone for spatiotemporal feature learning directly from a volumetric patch.) (D. The volumetric patches are processed with a pretrained feature encoder network. The resulting set of features is combined using a learnable attention module to produce a final patient-level prediction. )
• Figure 2: VCore Gleason GG classification results. (a AUC, F1, Precision and Recall for 5 class classification problem comparing volumetric, 2D and natural-image based features. ) (b Confusion matrix comparing actual and predicted labels for 5 class classification comparing volumetric, 2D, and natural-image-based features. ) (c AUC, F1, Precision and Recall for binary model performance (GGG $\ge$ 2 vs. GGG $<$ 2) comparing volumetric, 2D and natural-image based features. )
• Figure 3: Results of the clinical validation of the volumetric core using a digital pathology slide viewer with advanced functionality. (a Confusion matrix illustrating intra-observer variability for two pathologists (microscope assessment vs. digital assessment using the volumetric core) (b Confusion matrix illustrating inter-observer variability between two pathologists (microscope assessment and digital assessment using the volumetric core)). ASAP - atypical small acinar proliferation, HGPIN - high-grade prostatic intraepithelial neoplasia, PNI - perineural invasion, IDC - intraductal carcinoma.
• Figure 4: Visualization of the discriminative regions from WSIs overlaid with attention maps from ABMIL module from two clinically significant cores (Core A and Core B) where VCore resulted in a true positive (TP) prediction and 2D DINO resulted in a false negative (FN) prediction. White box indicates patches highlighted by pathologists.
• Figure 5: Visualization of the discriminative patches from WSIs overlaid with attention maps from the TimeSFormer model determined to be tumor patches containing volumetric information important to detecting clinical significance.