Histology Virtual Staining with Mask-Guided Adversarial Transfer Learning for Tertiary Lymphoid Structure Detection

TL;DR

This work tackles TLS detection in solid tumors by converting widely available H&E slides into virtual IHC-like patches to simulate CD20 staining. It introduces VIPA-Net, combining a Mask-Guided Adversarial Transfer Learning module with an H&E-based TLS detection module to leverage both real H&E and synthetic IHC information. The approach reduces reliance on explicit IHC labeling while improving TLS localization on WSIs, demonstrated on a TCGA-derived dataset with over 10,000 TLS annotations. Practically, this enables accurate TLS detection in settings where CD20 staining is unavailable or impractical, enhancing immunotherapy-relevant pathology analysis.

Abstract

Histological Tertiary Lymphoid Structures (TLSs) are increasingly recognized for their correlation with the efficacy of immunotherapy in various solid tumors. Traditionally, the identification and characterization of TLSs rely on immunohistochemistry (IHC) staining techniques, utilizing markers such as CD20 for B cells. Despite the specificity of IHC, Hematoxylin-Eosin (H&E) staining offers a more accessible and cost-effective choice. Capitalizing on the prevalence of H&E staining slides, we introduce a novel Mask-Guided Adversarial Transfer Learning method designed for virtual pathological staining. This method adeptly captures the nuanced color variations across diverse tissue types under various staining conditions, such as nucleus, red blood cells, positive reaction regions, without explicit label information, and adeptly synthesizes realistic IHC-like virtual staining patches, even replicating the positive reaction. Further, we propose the Virtual IHC Pathology Analysis Network (VIPA-Net), an integrated framework encompassing a Mask-Guided Transfer Module and an H&E-Based Virtual Staining TLS Detection Module. VIPA-Net synergistically harnesses both H\&E staining slides and the synthesized virtual IHC patches to enhance the detection of TLSs within H&E Whole Slide Images (WSIs). We evaluate the network with a comprehensive dataset comprising 1019 annotated slides from The Cancer Genome Atlas (TCGA). Experimental results compellingly illustrate that the VIPA-Net substantially elevates TLS detection accuracy, effectively circumventing the need for actual CD20 staining across the public dataset.

Figures (8)

• Figure 1: H&E Combined with CD20 Staining Improves The Detection Results. Our method significantly improves the detection of TLSs by accurately generating virtual CD20 staining patches, thereby reducing the false detection of TLSs.
• Figure 2: Mask-Guided Adversarial Transfer Learning. In this module, the left side contains the real CD20 staining patches, the virtual CD20 staining patches, and the corresponding automatically generated cell and stain masks. And the right side contains the real H&E staining patches, the virtual H&E staining patches, and the corresponding automatically generated nucleus and red blood cell masks.
• Figure 3: Unsupervised Mask Extraction. Extract the best-performing mask images from different channels of the staining patches. This process uses thresholds to automatically extract the masks, avoiding the time-consuming and labor-intensive manual annotation.
• Figure 4: The Detection Models input concatenation Process. We concatenate the RGB three-channel patches of H&E and CD20 into six-channel patches and then input them into the detection model to obtain their respective predicted results, which are shown with yellow candidate boxes.
• Figure 5: The Dataset Visualization. For the TCGA dataset, we compare the effect of H&E staining patches and mask-guided CD20 staining patches. In the images, the left side of the green dividing line shows patches with the presence of TLS, while the right side displays patches without TLS. The green mask represents the ground truth of TLS.