Mix-Domain Contrastive Learning for Unpaired H&E-to-IHC Stain Translation

TL;DR

The proposed Mix-Domain Contrastive Learning (MDCL) method aggregates the inter-domain and intra-domain pathology information by estimating the correlation between the anchor patch and all the patches from the matching images, encouraging the network to learn additional contrastive knowledge from mixed domains.

Abstract

H&E-to-IHC stain translation techniques offer a promising solution for precise cancer diagnosis, especially in low-resource regions where there is a shortage of health professionals and limited access to expensive equipment. Considering the pixel-level misalignment of H&E-IHC image pairs, current research explores the pathological consistency between patches from the same positions of the image pair. However, most of them overemphasize the correspondence between domains or patches, overlooking the side information provided by the non-corresponding objects. In this paper, we propose a Mix-Domain Contrastive Learning (MDCL) method to leverage the supervision information in unpaired H&E-to-IHC stain translation. Specifically, the proposed MDCL method aggregates the inter-domain and intra-domain pathology information by estimating the correlation between the anchor patch and all the patches from the matching images, encouraging the network to learn additional contrastive knowledge from mixed domains. With the mix-domain pathology information aggregation, MDCL enhances the pathological consistency between the corresponding patches and the component discrepancy of the patches from the different positions of the generated IHC image. Extensive experiments on two H&E-to-IHC stain translation datasets, namely MIST and BCI, demonstrate that the proposed method achieves state-of-the-art performance across multiple metrics.

Figures (5)

• Figure 1: Some examples of H&E-IHC image pairs from public MIST li2023adaptive and BCI liu2022bci datasets. Matching H&E-IHC images are unpaired at the pixel level.
• Figure 2: The difference between (a) existing methods and (b) MDCL. For each anchor patch, existing methods only consider inter-domain patches (red and green). MDCL utilizes patches from both inter- (red and green) and intra-domain (blue).
• Figure 3: H&E-to-IHC stain translation learning framework. Given the input H&E stained image $I_{he}$, the generator $G$ generates the virtual IHC stained image $I_{ihc}^v$. $M$ patches from $I_{ihc}^v$ are randomly sampled to construct the anchor set $A$. At the same locations, the positive patches $p^{he}_i$ from $I_{he}$ and $p^{gt}_i$ from $I_{ihc}^{gt}$ are selected to build $P^{he}$ and $P^{gt}$. The Siamese network $f$ maps each pair of $(a_i, p_i)$ together under the constraint of the mix-domain contrastive loss.
• Figure 4: Mix-domain contrastive learning for the anchor set $A$ and positive set $P^{he}$. All the patches from $A$ and $P^{he}$ are at first mapped onto the unit hypersphere by the Siamese network. For each anchor $a_i$, its embedding $z_i$ is demanded to be close to $z_i^{he}$ (the embedding of the positive $p_i^{he}$) and far away from all the rest.
• Figure 5: Visualization results of some H&E-IHC image pairs. (a) Original images, including input H&E images, GroundTruth IHC images, and the generated IHC images with ASP and MDCL. (b) Input H&E patch, (c) Groundtruth IHC patch, (c) Generated IHC patch with ASP, (d) Generated IHC patch with MDCL.