VISTA-PATH: An interactive foundation model for pathology image segmentation and quantitative analysis in computational pathology

TL;DR

VISTA-PATH reframes pathology image segmentation as an interactive, class-aware process that combines visual context, semantic tissue prompts, and spatial priors to produce clinically meaningful pixel-level maps. It is trained on VISTA-PATH Data, a 1.6+ million triplet corpus spanning 9 organs and 93 tissue classes, enabling robust cross-organ and cross-domain generalization. The framework supports real-time human-in-the-loop refinement via bounding-box prompts, leveraging patch-level corrections to improve whole-slide segmentations with minimal annotation. Furthermore, segmentation outputs feed downstream analyses, exemplified by Tumor Interaction Score (TIS), which links morphology to patient survival in TCGA-COAD and outperforms traditional MIL-based approaches, underscoring the practical impact for digital pathology and prognostic modeling.

Abstract

Accurate semantic segmentation for histopathology image is crucial for quantitative tissue analysis and downstream clinical modeling. Recent segmentation foundation models have improved generalization through large-scale pretraining, yet remain poorly aligned with pathology because they treat segmentation as a static visual prediction task. Here we present VISTA-PATH, an interactive, class-aware pathology segmentation foundation model designed to resolve heterogeneous structures, incorporate expert feedback, and produce pixel-level segmentation that are directly meaningful for clinical interpretation. VISTA-PATH jointly conditions segmentation on visual context, semantic tissue descriptions, and optional expert-provided spatial prompts, enabling precise multi-class segmentation across heterogeneous pathology images. To support this paradigm, we curate VISTA-PATH Data, a large-scale pathology segmentation corpus comprising over 1.6 million image-mask-text triplets spanning 9 organs and 93 tissue classes. Across extensive held-out and external benchmarks, VISTA-PATH consistently outperforms existing segmentation foundation models. Importantly, VISTA-PATH supports dynamic human-in-the-loop refinement by propagating sparse, patch-level bounding-box annotation feedback into whole-slide segmentation. Finally, we show that the high-fidelity, class-aware segmentation produced by VISTA-PATH is a preferred model for computational pathology. It improve tissue microenvironment analysis through proposed Tumor Interaction Score (TIS), which exhibits strong and significant associations with patient survival. Together, these results establish VISTA-PATH as a foundation model that elevates pathology image segmentation from a static prediction to an interactive and clinically grounded representation for digital pathology. Source code and demo can be found at https://github.com/zhihuanglab/VISTA-PATH.

Figures (13)

• Figure 1: Overview of the VISTA-PATH Dataset and the VISTA-PATH architecturea, Data acquisition pipeline from public pathology repositories. b, Distribution of segmentation masks across organs in the VISTA-PATH Dataset (log scale). c, Constructed tissue ontology organizing class labels across organs; accompanying bar plots indicate the number of images containing each tissue class (log scale). d, VISTA-PATH model workflow. Given an H&E image, class prompts, and a bounding box, vision, text, and prompt encoders extract features that are fused via cross-attention modules. A mask decoder produces class-specific segmentation maps, yielding initial predictions. Pathology experts can provide real-time bounding-box feedback to refine segmentation results. e, Human-in-the-loop refinement workflow. The original H&E image produces pixel-wise segmentations from VISTA-PATH and patch-wise segmentations via a patch embedding model. A pathologist revises annotations on a small subset of patches. These refinements are used to learn a patch-level embedding classifier, which generalizes to infer bounding boxes across the entire image. The inferred bounding boxes are incorporated into VISTA-PATH to obtain improved pixel-level segmentation of the full image with minimal human intervention. Human-in-the-loop refinement interface is implement using TissueLab li2025co.
• Figure 1: Extended Data Figure 1. Overview of VISTA-PATH Data.a, Bar plots showing the distribution of tissue classes across organs in the VISTA-PATH Dataset, grouped by organ type. Counts are displayed on a logarithmic scale. b, Bubble plot summarizing the pathology tissue classes in VISTA-PATH Data, where each bubble represents a tissue class and its size is proportional to the number of annotated image--mask instances. c, Bubble plot of pathology tissue class distribution in the BiomedParse dataset, shown using the same visualization scheme as in b, Compared with VISTA-PATH, BiomedParse exhibits substantially fewer tissue categories and a more limited semantic coverage.
• Figure 2: Comparison of VISTA-PATH with existing segmentation foundation models.a--c, Architectural comparison of representative segmentation foundation models. BiomedParse (a) accepts an image and class-aware textual prompts, but lacks explicit spatial priors (e.g., bounding boxes). MedSAM (b) takes an image and class-specific bounding boxes as input to output binary segmentations; however, this design is prone to errors when multiple tissue types share overlapped bounding boxes. In contrast, VISTA-PATH (c) jointly leverages class-aware text prompts and corresponding bounding boxes to enable spatially informed, precise multi-class segmentation. d--f, Quantitative segmentation performance on the VISTA-PATH held-out test set. (d) Bar plots comparing performance across individual datasets. For each dataset, the number of images, number of classes, error bars (95% confidence intervals), and $P$ values are reported. Statistical significance is assessed using two-sided Student's $t$-test ($^{*}P < 0.05$; $^{**}P < 1 \times 10^{-2}$; $^{***}P < 1 \times 10^{-3}$). (e) Radar plot comparing model performance across $9$ organ types. (f) Line plots showing Dice scores with 95% confidence intervals across different organs, grouped by tumor-related, normal-related, and microenvironment-related tissue categories. g, Case studies illustrating representative segmentation results.
• Figure 2: Extended Data Figure 2. Overview of the held-out internal datasets used for evaluation.a, Sunburst diagram illustrating the hierarchical organization of tissue classes in the held-out internal evaluation datasets. Inner rings denote organ types, while outer rings correspond to pathology tissue labels. b, Bar plot showing the number of annotated image-mask instances for each organ in the held-out internal evaluation datasets. c, Sankey diagram visualizing the hierarchical semantic structure of tissue types in the held-out internal datasets. Flow width is proportional to the number of samples, with sample counts shown in parentheses.
• Figure 3: VISTA-PATH generalizes across annotation protocols, organs, and tissue types on external datasets.a, Segmentation evaluation of LungHP and OCDC datasets. b, Segmentation evaluation of 5 Visium HD datasets. c, Segmentation evaluation of 22 Xenium datasets. d, Radar plot comparing model performance across 13 organ types. e, Line plots showing Dice scores with 95% confidence intervals across different organs, grouped by tumor-related, microenvironment-related, and normal anatomical tissue categories.