Leveraging Adversarial Learning for Pathological Fidelity in Virtual Staining

TL;DR

This work tackles the problem of generating pathologically faithful immunohistochemical (IHC) stains from H&E sections to reduce reliance on costly IHC procedures. It introduces CSSP2P GAN, a conditional GAN where the discriminator is conditioned on both H&E and IHC to preserve semantic information, and systematically ablates components like Pyramid $L_1$ loss and a CSS loss $L_{ ext{CSS}}$ to improve fidelity. Using the HER2match dataset with well-registered same-section H&E–HER2 IHC tile pairs, the study demonstrates that adversarial loss and H&E conditioning enhance semantic similarity (lower FID/KID, higher MS-SSIM) and that CSSP2P achieves superior pathological fidelity as judged by blind pathologist evaluation, compared to Pyramid Pix2Pix and ASP baselines. The findings also reveal that traditional metrics such as SSIM and PSNR may not reliably reflect virtual staining quality, underscoring the need for robust evaluation metrics and more same-section datasets for clinical translation.

Abstract

In addition to evaluating tumor morphology using H&E staining, immunohistochemistry is used to assess the presence of specific proteins within the tissue. However, this is a costly and labor-intensive technique, for which virtual staining, as an image-to-image translation task, offers a promising alternative. Although recent, this is an emerging field of research with 64% of published studies just in 2024. Most studies use publicly available datasets of H&E-IHC pairs from consecutive tissue sections. Recognizing the training challenges, many authors develop complex virtual staining models based on conditional Generative Adversarial Networks, but ignore the impact of adversarial loss on the quality of virtual staining. Furthermore, overlooking the issues of model evaluation, they claim improved performance based on metrics such as SSIM and PSNR, which are not sufficiently robust to evaluate the quality of virtually stained images. In this paper, we developed CSSP2P GAN, which we demonstrate to achieve heightened pathological fidelity through a blind pathological expert evaluation. Furthermore, while iteratively developing our model, we study the impact of the adversarial loss and demonstrate its crucial role in the quality of virtually stained images. Finally, while comparing our model with reference works in the field, we underscore the limitations of the currently used evaluation metrics and demonstrate the superior performance of CSSP2P GAN.

Figures (1)

• Figure 1: Test set examples, comparing the outputs of Pyramid Pix2Pix, ASP, and CSSP2P GAN models, across the HER2-scores.