Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PathoHR: Breast Cancer Survival Prediction on High-Resolution Pathological Images

Yang Luo, Shiru Wang, Jun Liu, Jiaxuan Xiao, Rundong Xue, Zeyu Zhang, Hao Zhang, Yu Lu, Yang Zhao, Yutong Xie

TL;DR

Breast cancer survival prediction from whole slide images is hindered by tumor heterogeneity and the computational burden of high-resolution data. PathoHR presents a multi-resolution pipeline that combines a plug-and-play high-resolution Vision Transformer (ViT) with Adaptive Token Merge (ATM) and Fuzzy Positional Encoding (FPE) via the ViTAR encoder to enhance patch representations, while systematically evaluating token-merging similarity metrics. The approach demonstrates that smaller 16×16 patches, when processed through PathoHR, can outperform traditional larger-patch baselines (24×24) and reduce computational demands, validated on the TCGA-BRCA dataset for overall survival prediction. This work advances practical high-resolution computational pathology by enabling accurate, efficient OS prediction and lays groundwork for incorporating multi-modal data in the future.

Abstract

Breast cancer survival prediction in computational pathology presents a remarkable challenge due to tumor heterogeneity. For instance, different regions of the same tumor in the pathology image can show distinct morphological and molecular characteristics. This makes it difficult to extract representative features from whole slide images (WSIs) that truly reflect the tumor's aggressive potential and likely survival outcomes. In this paper, we present PathoHR, a novel pipeline for accurate breast cancer survival prediction that enhances any size of pathological images to enable more effective feature learning. Our approach entails (1) the incorporation of a plug-and-play high-resolution Vision Transformer (ViT) to enhance patch-wise WSI representation, enabling more detailed and comprehensive feature extraction, (2) the systematic evaluation of multiple advanced similarity metrics for comparing WSI-extracted features, optimizing the representation learning process to better capture tumor characteristics, (3) the demonstration that smaller image patches enhanced follow the proposed pipeline can achieve equivalent or superior prediction accuracy compared to raw larger patches, while significantly reducing computational overhead. Experimental findings valid that PathoHR provides the potential way of integrating enhanced image resolution with optimized feature learning to advance computational pathology, offering a promising direction for more accurate and efficient breast cancer survival prediction. Code will be available at https://github.com/AIGeeksGroup/PathoHR.

PathoHR: Breast Cancer Survival Prediction on High-Resolution Pathological Images

TL;DR

Breast cancer survival prediction from whole slide images is hindered by tumor heterogeneity and the computational burden of high-resolution data. PathoHR presents a multi-resolution pipeline that combines a plug-and-play high-resolution Vision Transformer (ViT) with Adaptive Token Merge (ATM) and Fuzzy Positional Encoding (FPE) via the ViTAR encoder to enhance patch representations, while systematically evaluating token-merging similarity metrics. The approach demonstrates that smaller 16×16 patches, when processed through PathoHR, can outperform traditional larger-patch baselines (24×24) and reduce computational demands, validated on the TCGA-BRCA dataset for overall survival prediction. This work advances practical high-resolution computational pathology by enabling accurate, efficient OS prediction and lays groundwork for incorporating multi-modal data in the future.

Abstract

Breast cancer survival prediction in computational pathology presents a remarkable challenge due to tumor heterogeneity. For instance, different regions of the same tumor in the pathology image can show distinct morphological and molecular characteristics. This makes it difficult to extract representative features from whole slide images (WSIs) that truly reflect the tumor's aggressive potential and likely survival outcomes. In this paper, we present PathoHR, a novel pipeline for accurate breast cancer survival prediction that enhances any size of pathological images to enable more effective feature learning. Our approach entails (1) the incorporation of a plug-and-play high-resolution Vision Transformer (ViT) to enhance patch-wise WSI representation, enabling more detailed and comprehensive feature extraction, (2) the systematic evaluation of multiple advanced similarity metrics for comparing WSI-extracted features, optimizing the representation learning process to better capture tumor characteristics, (3) the demonstration that smaller image patches enhanced follow the proposed pipeline can achieve equivalent or superior prediction accuracy compared to raw larger patches, while significantly reducing computational overhead. Experimental findings valid that PathoHR provides the potential way of integrating enhanced image resolution with optimized feature learning to advance computational pathology, offering a promising direction for more accurate and efficient breast cancer survival prediction. Code will be available at https://github.com/AIGeeksGroup/PathoHR.

Paper Structure

This paper contains 11 sections, 6 equations, 3 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Method
  4. Patch-wise Feature Extraction
  5. Similarity Calculation for Token Merging
  6. Plug-in-play ViTAR Encoder
  7. Experiments
  8. Dataset and Evaluation Metrics
  9. Comparetive Study
  10. Ablation Study
  11. Conclusion

Figures (3)

  • Figure 1: The proposed PathoHR pipeline for breast cancer os prediction. The pipeline consists of three main components: (1) patch-wise feature extraction, (2) token merge similarity calculation for representation learning, and (3) a plug-and-play ViTAR encoder, that connects to the Transformer Encoder Block and incorporates Attention operations to generate predictive outputs.
  • Figure 2: This figure illustrates five different methods of calculating similarity: (1) Euclidean Similarity jones2022spatial; (2) Cosine Similarity khan2021similarity; (3) Attention Score he2022transformers; (4) Semantic Similarity tizhoosh2024image; and (5) ToMe Similarity bolya2022tome.
  • Figure 3: Performance on breast cancer classification task. Different models using WSIs as input for breast cancer classification tasks are evaluated. AUC values are reported.