PriorPath: Coarse-To-Fine Approach for Controlled De-Novo Pathology Semantic Masks Generation

TL;DR

This work tackles data scarcity and bias in digital pathology by introducing PriorPath, a coarse-to-fine image-to-image translation framework that generates detailed, controllable semantic tissue masks from coarse region inputs. By comparing paired (pix2pix) and unpaired (CycleGAN) I2I models, and by pairing coarse with fine masks, the approach achieves improved realism and coverage of the semantic mask space, outperforming previous methods like DEPAS and DCGAN. The authors extend the pipeline to photorealistic RGB image generation via pix2pixHD, demonstrating a complete, controllable path from coarse tissue layout to realistic histopathology images across multiple cancer types. This scalable framework supports AI development in computational pathology, with potential extensions to multilabel masks and diffusion-based methods for even richer synthetic data generation.

Abstract

Incorporating artificial intelligence (AI) into digital pathology offers promising prospects for automating and enhancing tasks such as image analysis and diagnostic processes. However, the diversity of tissue samples and the necessity for meticulous image labeling often result in biased datasets, constraining the applicability of algorithms trained on them. To harness synthetic histopathological images to cope with this challenge, it is essential not only to produce photorealistic images but also to be able to exert control over the cellular characteristics they depict. Previous studies used methods to generate, from random noise, semantic masks that captured the spatial distribution of the tissue. These masks were then used as a prior for conditional generative approaches to produce photorealistic histopathological images. However, as with many other generative models, this solution exhibits mode collapse as the model fails to capture the full diversity of the underlying data distribution. In this work, we present a pipeline, coined PriorPath, that generates detailed, realistic, semantic masks derived from coarse-grained images delineating tissue regions. This approach enables control over the spatial arrangement of the generated masks and, consequently, the resulting synthetic images. We demonstrated the efficacy of our method across three cancer types, skin, prostate, and lung, showcasing PriorPath's capability to cover the semantic mask space and to provide better similarity to real masks compared to previous methods. Our approach allows for specifying desired tissue distributions and obtaining both photorealistic masks and images within a single platform, thus providing a state-of-the-art, controllable solution for generating histopathological images to facilitate AI for computational pathology.

Figures (4)

• Figure 1: Analysis and Illustration of Generative Approaches for High-Resolution Binary Semantic Masks of Tissue Structure. (a) DCGAN approach: Generates synthetic histopathological fine semantic masks from random noise, requiring real histopathological semantic masks for discriminator processing during training. (b) DEAPS method (current SOTA): Enhances DCGAN with discrete adaptive block final activation, multi-scale discriminators, and spatial noise within hidden layers. (c) PriorPath approach (this work): Produces complex, realistic semantic masks from coarse-grained semantic masks of regions of interest. (d) Demonstration of DEPAS mode collapse: t-SNE projection of Inception’s representations for the DEPAS method (blue) and the Real masks (black). The dataset includes three cancer types with H&E staining: Skin Cutaneous Melanoma (SKCM), Prostate Adenocarcinoma (PRAD), and Lung Squamous Cell Carcinoma (LUSC), along with a Non-small cell lung carcinoma (NSCLC) dataset with PD-L1 immunohistochemistry.
• Figure 2: Examples of PriorPath tissue mask generation compered with DEPAS. These figures show examples of four types of cancers, and three different organs: skin, prostate, and lung. For each realization, we show PriorPath fine-grain tissue mask generation from coarse-grain masks. The bottom panels show representative tissue masks taken from real biopsy patches, AKA Ground Truth (bottom, right), and by the current SOTA baseline, DEPAS (bottom, left). We show that PriorPath provides tissue masks from a distribution that is closer to the Ground Truth, and is superior in both quality and control of tissue mask generation rather than DEPAS’s outputs (as quantified in Table \ref{['tissue_mask_tab']}).
• Figure 3: Illustration of PriorPath results, demonstrating the benefits of controlling the distribution while preserving high similarity to the real distribution. (a) A t-SNE projection of inception’s representations for real skin cancer masks (black), DEPAS synthetic masks (blue), and PriorPath synthetic masks (red). PriorPath covers the real pathology mask space. The yellow line defines a grid of size 3 x 3 cells. (b) The similarity metric between the DEPAS and PriorPath synthetic masks and the real ones for each cell inside the grid of 3 x 3. The horizontal lines are the average similarity scores of the synthetic and real masks when taking into account all the masks (in all the grid's cells). (c) The local average similarity over the cell in a grid of size 3 x 3 for all four cancer realizations. The horizontal lines represent the global average similarity scores calculated over all masks (without dividing the representations into the grid).
• Figure 4: Full pipeline photorealistic results. Samples of fine-grain tissue masks and their corresponding histopathological photorealistic synthetic RGB images for all four types of cancer realizations. For each realization, pairs of real and synthetic masks and their corresponding images are shown. (a) Skin prostate tissues. (b) Lung tissues.