Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HECLIP: Histology-Enhanced Contrastive Learning for Imputation of Transcriptomics Profiles

Qing Wang, Wen-jie Chen, Bo Li, Jing Su, Guangyu Wang, Qianqian Song

TL;DR

HECLIP tackles the challenge of obtaining spatial gene expression without costly spatial transcriptomics by predicting transcriptomic profiles directly from H&E images. It introduces an image-centric contrastive learning framework that aligns patch-level histology features (via a ResNet-50 image encoder) with spatial gene expression representations through shared embeddings, and uses top-K retrieval to impute expressions for new patches. Across datasets such as GSE240429, GSE245620, and spatialLIBD, HECLIP achieves superior $RMSE$ and $SSIM$ and higher Hit@T than prior methods, with ablations confirming the benefit of the unimodal loss and data augmentation. The approach is scalable and cost-efficient, enabling molecular insights from routine histology and advancing precision medicine.

Abstract

Histopathology, particularly hematoxylin and eosin (H\&E) staining, plays a critical role in diagnosing and characterizing pathological conditions by highlighting tissue morphology. However, H\&E-stained images inherently lack molecular information, requiring costly and resource-intensive methods like spatial transcriptomics to map gene expression with spatial resolution. To address these challenges, we introduce HECLIP (Histology-Enhanced Contrastive Learning for Imputation of Profiles), an innovative deep learning framework that bridges the gap between histological imaging and molecular profiling. HECLIP is specifically designed to infer gene expression profiles directly from H\&E-stained images, eliminating the need for expensive spatial transcriptomics assays. HECLIP leverages an advanced image-centric contrastive loss function to optimize image representation learning, ensuring that critical morphological patterns in histology images are effectively captured and translated into accurate gene expression profiles. This design enhances the predictive power of the image modality while minimizing reliance on gene expression data. Through extensive benchmarking on publicly available datasets, HECLIP demonstrates superior performance compared to existing approaches, delivering robust and biologically meaningful predictions. Detailed ablation studies further underscore its effectiveness in extracting molecular insights from histology images. Additionally, HECLIP's scalable and cost-efficient approach positions it as a transformative tool for both research and clinical applications, driving advancements in precision medicine. The source code for HECLIP is openly available at https://github.com/QSong-github/HECLIP.

HECLIP: Histology-Enhanced Contrastive Learning for Imputation of Transcriptomics Profiles

TL;DR

HECLIP tackles the challenge of obtaining spatial gene expression without costly spatial transcriptomics by predicting transcriptomic profiles directly from H&E images. It introduces an image-centric contrastive learning framework that aligns patch-level histology features (via a ResNet-50 image encoder) with spatial gene expression representations through shared embeddings, and uses top-K retrieval to impute expressions for new patches. Across datasets such as GSE240429, GSE245620, and spatialLIBD, HECLIP achieves superior and and higher Hit@T than prior methods, with ablations confirming the benefit of the unimodal loss and data augmentation. The approach is scalable and cost-efficient, enabling molecular insights from routine histology and advancing precision medicine.

Abstract

Histopathology, particularly hematoxylin and eosin (H\&E) staining, plays a critical role in diagnosing and characterizing pathological conditions by highlighting tissue morphology. However, H\&E-stained images inherently lack molecular information, requiring costly and resource-intensive methods like spatial transcriptomics to map gene expression with spatial resolution. To address these challenges, we introduce HECLIP (Histology-Enhanced Contrastive Learning for Imputation of Profiles), an innovative deep learning framework that bridges the gap between histological imaging and molecular profiling. HECLIP is specifically designed to infer gene expression profiles directly from H\&E-stained images, eliminating the need for expensive spatial transcriptomics assays. HECLIP leverages an advanced image-centric contrastive loss function to optimize image representation learning, ensuring that critical morphological patterns in histology images are effectively captured and translated into accurate gene expression profiles. This design enhances the predictive power of the image modality while minimizing reliance on gene expression data. Through extensive benchmarking on publicly available datasets, HECLIP demonstrates superior performance compared to existing approaches, delivering robust and biologically meaningful predictions. Detailed ablation studies further underscore its effectiveness in extracting molecular insights from histology images. Additionally, HECLIP's scalable and cost-efficient approach positions it as a transformative tool for both research and clinical applications, driving advancements in precision medicine. The source code for HECLIP is openly available at https://github.com/QSong-github/HECLIP.
Paper Structure (17 sections, 7 equations, 4 figures, 3 tables, 1 algorithm)

This paper contains 17 sections, 7 equations, 4 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Method
  3. Datasets and Preprocessing
  4. Model input
  5. Embedding modules
  6. Loss function
  7. Inference stage
  8. Evaluation Metrics
  9. Results
  10. Benchmarking experiments demonstrate superior performance of HECLIP
  11. Loss convergence and embedding visualization
  12. HECLIP accurately predicts biologically important genes
  13. Ablation experiments of the HECLIP model
  14. Conclusions
  15. Competing interests
  16. ...and 2 more sections

Figures (4)

  • Figure 1: Overview of the HECLIP framework for transcriptomics imputation from histological images.
  • Figure 2: Comparison of methods for predicting transcriptomics from histology images based on RMSE metrics.
  • Figure 3: Comparison of loss convergence across different methods in training stage.
  • Figure 4: UMAP of the bi-modality embeddings from HECLIP and BLEEP. The blue dots are the Reference set and the orange dots are the Query set.