Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Advancing H&E-to-IHC Stain Translation in Breast Cancer: A Multi-Magnification and Attention-Based Approach

Linhao Qu, Chengsheng Zhang, Guihui Li, Haiyong Zheng, Chen Peng, Wei He

TL;DR

This work addresses the challenge of estimating HER2 status by translating H&E slides to IHC-HER2, reducing reliance on costly immunohistochemistry. It introduces a GAN-based framework that integrates multi-magnification processing with an attention-based feature interaction module (AMMFI) within the generator, along with a PatchGAN discriminator. The training leverages a composite loss combining adversarial, adaptive supervised contrastive (Adaptive Weighted PatchNCE), and Gaussian losses, with explicit weighting to balance fidelity and distribution alignment. Experiments on the MIST-HER2 breast cancer dataset show the proposed method outperforms state-of-the-art baselines across both paired and unpaired metrics, and ablation confirms the value of each component. The approach has the potential to improve the cost-efficiency and accessibility of biomarker assessment in breast cancer, potentially informing treatment decisions and patient outcomes.

Abstract

Breast cancer presents a significant healthcare challenge globally, demanding precise diagnostics and effective treatment strategies, where histopathological examination of Hematoxylin and Eosin (H&E) stained tissue sections plays a central role. Despite its importance, evaluating specific biomarkers like Human Epidermal Growth Factor Receptor 2 (HER2) for personalized treatment remains constrained by the resource-intensive nature of Immunohistochemistry (IHC). Recent strides in deep learning, particularly in image-to-image translation, offer promise in synthesizing IHC-HER2 slides from H\&E stained slides. However, existing methodologies encounter challenges, including managing multiple magnifications in pathology images and insufficient focus on crucial information during translation. To address these issues, we propose a novel model integrating attention mechanisms and multi-magnification information processing. Our model employs a multi-magnification processing strategy to extract and utilize information from various magnifications within pathology images, facilitating robust image translation. Additionally, an attention module within the generative network prioritizes critical information for image distribution translation while minimizing less pertinent details. Rigorous testing on a publicly available breast cancer dataset demonstrates superior performance compared to existing methods, establishing our model as a state-of-the-art solution in advancing pathology image translation from H&E to IHC staining.

Advancing H&E-to-IHC Stain Translation in Breast Cancer: A Multi-Magnification and Attention-Based Approach

TL;DR

This work addresses the challenge of estimating HER2 status by translating H&E slides to IHC-HER2, reducing reliance on costly immunohistochemistry. It introduces a GAN-based framework that integrates multi-magnification processing with an attention-based feature interaction module (AMMFI) within the generator, along with a PatchGAN discriminator. The training leverages a composite loss combining adversarial, adaptive supervised contrastive (Adaptive Weighted PatchNCE), and Gaussian losses, with explicit weighting to balance fidelity and distribution alignment. Experiments on the MIST-HER2 breast cancer dataset show the proposed method outperforms state-of-the-art baselines across both paired and unpaired metrics, and ablation confirms the value of each component. The approach has the potential to improve the cost-efficiency and accessibility of biomarker assessment in breast cancer, potentially informing treatment decisions and patient outcomes.

Abstract

Breast cancer presents a significant healthcare challenge globally, demanding precise diagnostics and effective treatment strategies, where histopathological examination of Hematoxylin and Eosin (H&E) stained tissue sections plays a central role. Despite its importance, evaluating specific biomarkers like Human Epidermal Growth Factor Receptor 2 (HER2) for personalized treatment remains constrained by the resource-intensive nature of Immunohistochemistry (IHC). Recent strides in deep learning, particularly in image-to-image translation, offer promise in synthesizing IHC-HER2 slides from H\&E stained slides. However, existing methodologies encounter challenges, including managing multiple magnifications in pathology images and insufficient focus on crucial information during translation. To address these issues, we propose a novel model integrating attention mechanisms and multi-magnification information processing. Our model employs a multi-magnification processing strategy to extract and utilize information from various magnifications within pathology images, facilitating robust image translation. Additionally, an attention module within the generative network prioritizes critical information for image distribution translation while minimizing less pertinent details. Rigorous testing on a publicly available breast cancer dataset demonstrates superior performance compared to existing methods, establishing our model as a state-of-the-art solution in advancing pathology image translation from H&E to IHC staining.
Paper Structure (16 sections, 7 equations, 3 figures, 2 tables)

This paper contains 16 sections, 7 equations, 3 figures, 2 tables.

Table of Contents

  1. INTRODUCTION
  2. Related Work
  3. Method
  4. Proposed Generator Overview
  5. Multi-magnification Processing Strategy
  6. Attention-based Multi-magnification Feature Interaction Module
  7. Training Loss Functions
  8. Experiments
  9. Datasets
  10. Implementation Details
  11. Evaluation Metrics
  12. Competitors
  13. Qualitative Evaluations
  14. Quantitative Evaluations
  15. Ablation Study
  16. ...and 1 more sections

Figures (3)

  • Figure 1: (a) Our proposed multi-magnification pathology image processing strategy. (b) The overall structure of the proposed generator. (c) AMMFI refers to the attention-based multi-magnification feature interaction module. Conv denotes a 3x3 convolutional layer, while BN signifies a batch normalization layer. BSRGAN is identified as a super-resolution model. Mul. indicates an element-wise multiplication operation, and Cat represents concatenation. CA is an abbreviation for channel attention, and SA stands for spatial attention.
  • Figure 2: Visual results of our method and the competitors. "H&E_input" and "IHC_real" respectively represent the input image stained with H&E and the real image stained with IHC.
  • Figure 3: Visual results of the ablation study. "H&E_input" and "IHC_real" respectively represent the input image stained with H&E and the real image stained with IHC. "Our Baseline" means that our proposed attention module and multi-magnification strategy are not used. "w/. Attention" and "w/. multi-magnification" mean that only our proposed attention module or multi-magnification strategy is used respectively. "Ours" means that both of the strategies are used.