Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dual-attention ResNet outperforms transformers in HER2 prediction on DCE-MRI

Naomi Fridman, Anat Goldstein

TL;DR

This study tackles noninvasive prediction of HER2 status from dynamic contrast-enhanced MRI (DCE-MRI) and the challenge of converting high-dynamic-range data into 8-bit RGB inputs for pretrained networks. It introduces the Triple-Head Dual-Attention ResNet (THDA-ResNet), which processes RGB-fused temporal phases with dual attention to capture transferable spatiotemporal features. On a multicenter I-SPY dataset (n=1,149) THDA-ResNet outperforms transformer baselines, achieving AUC around 0.74, while transformer models lag in AUC and sensitivity, and N4 bias-field correction tends to degrade performance; external validation on AMBL (n=43) shows moderate generalizability (AUC 0.61–0.66) under domain shift. The findings highlight the importance of data presentation and preprocessing choices, with per-channel normalization and upper percentile clipping yielding robust HER2 prediction, supporting broader use of reproducible DL biomarkers in breast cancer imaging.

Abstract

Breast cancer is the most diagnosed cancer in women, with HER2 status critically guiding treatment decisions. Noninvasive prediction of HER2 status from dynamic contrast-enhanced MRI (DCE-MRI) could streamline diagnostics and reduce reliance on biopsy. However, preprocessing high-dynamic-range DCE-MRI into standardized 8-bit RGB format for pretrained neural networks is nontrivial, and normalization strategy significantly affects model performance. We benchmarked intensity normalization strategies using a Triple-Head Dual-Attention ResNet that processes RGB-fused temporal sequences from three DCE phases. Trained on a multicenter cohort (n=1,149) from the I-SPY trials and externally validated on BreastDCEDL_AMBL (n=43 lesions), our model outperformed transformer-based architectures, achieving 0.75 accuracy and 0.74 AUC on I-SPY test data. N4 bias field correction slightly degraded performance. Without fine-tuning, external validation yielded 0.66 AUC, demonstrating cross-institutional generalizability. These findings highlight the effectiveness of dual-attention mechanisms in capturing transferable spatiotemporal features for HER2 stratification, advancing reproducible deep learning biomarkers in breast cancer imaging.

Dual-attention ResNet outperforms transformers in HER2 prediction on DCE-MRI

TL;DR

This study tackles noninvasive prediction of HER2 status from dynamic contrast-enhanced MRI (DCE-MRI) and the challenge of converting high-dynamic-range data into 8-bit RGB inputs for pretrained networks. It introduces the Triple-Head Dual-Attention ResNet (THDA-ResNet), which processes RGB-fused temporal phases with dual attention to capture transferable spatiotemporal features. On a multicenter I-SPY dataset (n=1,149) THDA-ResNet outperforms transformer baselines, achieving AUC around 0.74, while transformer models lag in AUC and sensitivity, and N4 bias-field correction tends to degrade performance; external validation on AMBL (n=43) shows moderate generalizability (AUC 0.61–0.66) under domain shift. The findings highlight the importance of data presentation and preprocessing choices, with per-channel normalization and upper percentile clipping yielding robust HER2 prediction, supporting broader use of reproducible DL biomarkers in breast cancer imaging.

Abstract

Breast cancer is the most diagnosed cancer in women, with HER2 status critically guiding treatment decisions. Noninvasive prediction of HER2 status from dynamic contrast-enhanced MRI (DCE-MRI) could streamline diagnostics and reduce reliance on biopsy. However, preprocessing high-dynamic-range DCE-MRI into standardized 8-bit RGB format for pretrained neural networks is nontrivial, and normalization strategy significantly affects model performance. We benchmarked intensity normalization strategies using a Triple-Head Dual-Attention ResNet that processes RGB-fused temporal sequences from three DCE phases. Trained on a multicenter cohort (n=1,149) from the I-SPY trials and externally validated on BreastDCEDL_AMBL (n=43 lesions), our model outperformed transformer-based architectures, achieving 0.75 accuracy and 0.74 AUC on I-SPY test data. N4 bias field correction slightly degraded performance. Without fine-tuning, external validation yielded 0.66 AUC, demonstrating cross-institutional generalizability. These findings highlight the effectiveness of dual-attention mechanisms in capturing transferable spatiotemporal features for HER2 stratification, advancing reproducible deep learning biomarkers in breast cancer imaging.
Paper Structure (23 sections, 6 equations, 3 figures, 4 tables)

This paper contains 23 sections, 6 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Study Population
  4. DCE-MRI Preprocessing and Normalization
  5. Per-slice min-max normalization.
  6. Global min-max normalization.
  7. Lower quantile clipping.
  8. Channelwise upper clipping.
  9. Global upper clipping.
  10. N4 bias field correction.
  11. Preprocessing Strategy Evaluation for HER2 Prediction
  12. Deep Learning Modeling
  13. Triple-Head Dual-Attention ResNet Architecture
  14. Comparative Architectures
  15. Patient-Level Prediction Pipeline
  16. ...and 8 more sections

Figures (3)

  • Figure 1: Intensity normalization strategies for DCE-MRI (dynamic contrast-enhanced magnetic resonance imaging) preprocessing. (a) Intensity distributions before (left) and after (right) bias field correction, showing the three temporal acquisitions (pre-contrast, post-early, post-late) with the 0.99 quantile threshold marked. The threshold value is unitless, indicating a dimensionless quantity. (b) Raw RGBfused images demonstrating four normalization approaches: global min-max normalization, per-slice min-max normalization, $q_{0.95}$ upper clipping, and $q_{0.99}$ upper clipping. (c) Corresponding bias-corrected images showing improved contrast consistency and reduced intensity variations across the four normalization strategies ($n=10$), including a scale bar representing 100 $\mu$m.
  • Figure 2: Triple-Head Dual-Attention ResNet architecture for DCE-MRI analysis. (a) Overall architecture showing per-channel processing of RGB-fused temporal phases through independent ResNet34 backbones with multi-scale spatial attention mechanisms, followed by channel fusion and classification. (b) Multi-scale spatial attention module detail demonstrating feature extraction from ResNet Layer3 and Layer4, spatial attention generation, element-wise multiplication for feature refinement, and multi-scale fusion through global average pooling and concatenation.
  • Figure 3: Triple-Head Dual-Attention ResNet mechanism visualization for DCE-MRI (dynamic contrast-enhanced magnetic resonance imaging) temporal fusion. (a) Input decomposition showing the original RGB-fused DCE-MRI image and its constituent channels (red: precontrast, green: early post-contrast, blue: late post-contrast), with learned channel attention weights indicating relative importance of each temporal phase. (b) Spatial attention maps generated by the multi-scale attention module (Layer3, 14×14 resolution) for each temporal channel, highlighting tumor-relevant regions; combined spatial attention demonstrates consensus across temporal phases. (c) Weighted fusion process showing attention-modulated individual channels and their integration into the final fused representation.