PathOrchestra: A Comprehensive Foundation Model for Computational Pathology with Over 100 Diverse Clinical-Grade Tasks

TL;DR

Overall, PathOrchestra demonstrates the clinical readiness of large-scale self-supervised pathology foundation models, achieving high accuracy and offering potential to digital medicine integration.

Abstract

The complexity and variability inherent in high-resolution pathological images present significant challenges in computational pathology. While pathology foundation models leveraging AI have catalyzed transformative advancements, their development demands large-scale datasets, considerable storage capacity, and substantial computational resources. Furthermore, ensuring their clinical applicability and generalizability requires rigorous validation across a broad spectrum of clinical tasks. Here, we present PathOrchestra, a versatile pathology foundation model trained via self-supervised learning on a dataset comprising 300K pathological slides from 20 tissue and organ types across multiple centers. The model was rigorously evaluated on 112 clinical tasks using a combination of 61 private and 51 public datasets. These tasks encompass digital slide preprocessing, pan-cancer classification, lesion identification, multi-cancer subtype classification, biomarker assessment, gene expression prediction, and the generation of structured reports. PathOrchestra demonstrated exceptional performance across 27,755 WSIs and 9,415,729 ROIs, achieving over 0.950 accuracy in 47 tasks, including pan-cancer classification across various organs, lymphoma subtype diagnosis, and bladder cancer screening. Notably, it is the first model to generate structured reports for high-incidence colorectal cancer and diagnostically complex lymphoma-areas that are infrequently addressed by foundational models but hold immense clinical potential. Overall, PathOrchestra exemplifies the feasibility and efficacy of a large-scale, self-supervised pathology foundation model, validated across a broad range of clinical-grade tasks. Its high accuracy and reduced reliance on extensive data annotation underline its potential for clinical integration, offering a pathway toward more efficient and high-quality medical services.

Figures (12)

• Figure 1: Overview of PathOrchestra. (Caption continued on next page.)
• Figure 2: (Continued) a. The foundational model was trained on 300K WSIs sourced from three centers, covering 20 human organs and tissues. b. The training strategy followed an unsupervised contrastive learning framework, utilizing a teacher model as the foundation for testing downstream tasks. c. Diverse downstream task data encompasses both histology and cytology, enabling the model to support 112 clinical tasks across seven major categories.
• Figure 3: Overview of Downstream Tasks and Dataset Statistics. It covers a total of 112 tasks, including 61 WSI tasks and 51 ROI tasks, with a vast dataset comprising 27,755 WSIs and 9,415,729 ROIs. The datasets used span both public and private sources, with 51 public datasets and 61 private datasets, incorporating over 35 types of stains and analyzing more than 50 genes. The downstream tasks are categorized into seven groups: Slide Preprocessing (12 tasks), Pan-Cancer (3 tasks), Lesion Identification (15 tasks), Cancer Subtyping (36 tasks), Biomarker Evaluation (36 tasks), and Gene Expression Assessment (10 tasks). The methodology employed for these tasks includes detection, retrieval, segmentation, linear classification, and weakly supervised classification approaches, utilizing models like ViTDet li2022exploring, KNN peterson2009k, Mask2Former cheng2022masked, Linear Probing he2022masked, and ABMIL ilse2018attention.
• Figure 4: Performance on Pathology Image Preprocessing and Quality Control Tasks. The model's performance in quality control and general analysis tasks included 12 ROI-level classification sub-tasks, covering areas such as pathology image quality control, magnification recognition, staining type identification, specimen collection method identification, and slide preparation form identification.
• Figure 5: Results of Pan-cancer Classification Tasks. a. The performance of the model in weakly supervised WSI-level classification tasks for 17 cancer types using in-house center data is illustrated. The bar chart demonstrates the impact of the model on these predictions. b and c. Data from 32 high-incidence cancer types were selected from the TCGA database to evaluate the model's performance in pan-cancer prediction across two slide formats including FFPE and Frozen. The bar charts depict the model's impact on prediction accuracy for each format.