Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive H&E-IHC information fusion staining framework based on feature extra

Yifan Jia, Xingda Yu, Zhengyang Ji, Songning Lai, Yutao Yue

TL;DR

This work addresses the challenge of cost-effective H&E-to-IHC virtual staining in histopathology by mitigating information loss and groundtruth limitations through an adaptive, information-enhanced framework. It introduces a VMFE-based multi-scale feature extractor, dual encoders trained with contrastive learning to align H&E and IHC representations, cross-attention fusion to guide generation, and an adaptive L1 loss to handle asymmetry between modalities. Experimental validation on the BCI Challenge and MIST datasets demonstrates competitive PSNR/SSIM and robust detail preservation, with ablations confirming the contribution of each component. The approach offers a scalable, clinically relevant pathway toward rapid, accurate IHC visualization from H&E images, with potential impact on diagnostic workflows.

Abstract

Immunohistochemistry (IHC) staining plays a significant role in the evaluation of diseases such as breast cancer. The H&E-to-IHC transformation based on generative models provides a simple and cost-effective method for obtaining IHC images. Although previous models can perform digital coloring well, they still suffer from (i) coloring only through the pixel features that are not prominent in HE, which is easy to cause information loss in the coloring process; (ii) The lack of pixel-perfect H&E-IHC groundtruth pairs poses a challenge to the classical L1 loss.To address the above challenges, we propose an adaptive information enhanced coloring framework based on feature extractors. We first propose the VMFE module to effectively extract the color information features using multi-scale feature extraction and wavelet transform convolution, while combining the shared decoder for feature fusion. The high-performance dual feature extractor of H&E-IHC is trained by contrastive learning, which can effectively perform feature alignment of HE-IHC in high latitude space. At the same time, the trained feature encoder is used to enhance the features and adaptively adjust the loss in the HE section staining process to solve the problems related to unclear and asymmetric information. We have tested on different datasets and achieved excellent performance.Our code is available at https://github.com/babyinsunshine/CEFF

Adaptive H&E-IHC information fusion staining framework based on feature extra

TL;DR

This work addresses the challenge of cost-effective H&E-to-IHC virtual staining in histopathology by mitigating information loss and groundtruth limitations through an adaptive, information-enhanced framework. It introduces a VMFE-based multi-scale feature extractor, dual encoders trained with contrastive learning to align H&E and IHC representations, cross-attention fusion to guide generation, and an adaptive L1 loss to handle asymmetry between modalities. Experimental validation on the BCI Challenge and MIST datasets demonstrates competitive PSNR/SSIM and robust detail preservation, with ablations confirming the contribution of each component. The approach offers a scalable, clinically relevant pathway toward rapid, accurate IHC visualization from H&E images, with potential impact on diagnostic workflows.

Abstract

Immunohistochemistry (IHC) staining plays a significant role in the evaluation of diseases such as breast cancer. The H&E-to-IHC transformation based on generative models provides a simple and cost-effective method for obtaining IHC images. Although previous models can perform digital coloring well, they still suffer from (i) coloring only through the pixel features that are not prominent in HE, which is easy to cause information loss in the coloring process; (ii) The lack of pixel-perfect H&E-IHC groundtruth pairs poses a challenge to the classical L1 loss.To address the above challenges, we propose an adaptive information enhanced coloring framework based on feature extractors. We first propose the VMFE module to effectively extract the color information features using multi-scale feature extraction and wavelet transform convolution, while combining the shared decoder for feature fusion. The high-performance dual feature extractor of H&E-IHC is trained by contrastive learning, which can effectively perform feature alignment of HE-IHC in high latitude space. At the same time, the trained feature encoder is used to enhance the features and adaptively adjust the loss in the HE section staining process to solve the problems related to unclear and asymmetric information. We have tested on different datasets and achieved excellent performance.Our code is available at https://github.com/babyinsunshine/CEFF

Paper Structure

This paper contains 17 sections, 7 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Multi-Scale Feature Extraction and Fusion
  4. Contrastive Learning Strategy of Dual Encoders
  5. Cross-Attention Feature Fusion between Encoder and Generator
  6. Adaptive L1 Loss
  7. Experiments
  8. Experimental Setup
  9. Datasets
  10. Experimental Details
  11. Evaluation Methods
  12. Comparative Experiments
  13. The performance of the dual encoder.
  14. Comparison with State-of-the-arts.
  15. Ablation Experiments
  16. ...and 2 more sections

Figures (3)

  • Figure 1: Overview of the proposed framework.
  • Figure 2: Encoder performance analysis on BCI dataset.
  • Figure 3: Visualize different methods on different dataset images.