USIGAN: Unbalanced Self-Information Feature Transport for Weakly Paired Image IHC Virtual Staining

TL;DR

USIGAN introduces unbalanced self-information transport to tackle weakly paired IHC virtual staining from H&E. By combining image-level cyclic unbalanced optimal transport (UOT-CTM) with intra-batch optical-density anchors (PC-SCM), the method prioritizes rare, diagnostically informative regions while removing weakly paired terms from distribution constraints. The framework delivers improved content fidelity and pathological semantic consistency, validated on MIST and IHC4BC with IoD, Pearson correlations, and perceptual metrics, and supported by extensive ablations and subjective pathologist evaluations. This approach enhances clinical relevance for virtual staining and offers a pathway toward efficient, pathology-aware cross-domain translation in digital pathology.

Abstract

Immunohistochemical (IHC) virtual staining is a task that generates virtual IHC images from H\&E images while maintaining pathological semantic consistency with adjacent slices. This task aims to achieve cross-domain mapping between morphological structures and staining patterns through generative models, providing an efficient and cost-effective solution for pathological analysis. However, under weakly paired conditions, spatial heterogeneity between adjacent slices presents significant challenges. This can lead to inaccurate one-to-many mappings and generate results that are inconsistent with the pathological semantics of adjacent slices. To address this issue, we propose a novel unbalanced self-information feature transport for IHC virtual staining, named USIGAN, which extracts global morphological semantics without relying on positional correspondence.By removing weakly paired terms in the joint marginal distribution, we effectively mitigate the impact of weak pairing on joint distributions, thereby significantly improving the content consistency and pathological semantic consistency of the generated results. Moreover, we design the Unbalanced Optimal Transport Consistency (UOT-CTM) mechanism and the Pathology Self-Correspondence (PC-SCM) mechanism to construct correlation matrices between H\&E and generated IHC in image-level and real IHC and generated IHC image sets in intra-group level.. Experiments conducted on two publicly available datasets demonstrate that our method achieves superior performance across multiple clinically significant metrics, such as IoD and Pearson-R correlation, demonstrating better clinical relevance.

Figures (12)

• Figure 1: Spatial heterogeneity between adjacent slices poses a major challenge in virtual staining. This heterogeneity manifests as misalignment of tissue structures and pathological semantics across slices, making it difficult to directly establish cross-domain mappings between morphology and staining styles.
• Figure 2: We leverage weakly paired IHC as an intermediate bridge to ensure global consistency between H&E and IHC while mining image-level self-information through a transport consistency framework. Simultaneously, we utilize focal optical density and DAB deconvolution to extract optical density feature vectors to guide intra-batch self-information mining. Multi-scale features within the batch are used as auxiliary features to fully explore intra-batch self-information.
• Figure 3: Strongly paired data (Cityspace dataset) exhibit a clear global matching relationship, while weakly paired data (MIST ER) demonstrate varying matching relationships influenced by spatial heterogeneity. In features reduced by PCA, the classical optimal transport approximates a one-to-one permutation structure, which is visualized through unified open source tools POT flamary2021pot. The orange points in the distribution of Weakly Paired IHC may contain multiple subgroups, but the matching patterns of these subgroups do not exhibit global consistency.
• Figure 4: Selected representative methods exhibit varying performances in virtual IHC staining results visualization on the MIST dataset. The quantitative comparison on different sate-of-art methods. Cell Segmentation and classification is performed using DeepLIIF ref_53 as follow by pati2024accelerating
• Figure 5: Selected representative methods exhibit varying performances in virtual IHC staining results visualization on the IHC4BC dataset. The quantitative comparison on different sate-of-art methods. Cell Segmentation and classification is performed using DeepLIIF ref_53 as follow by pati2024accelerating