Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cross-Slice Knowledge Transfer via Masked Multi-Modal Heterogeneous Graph Contrastive Learning for Spatial Gene Expression Inference

Zhiceng Shi, Changmiao Wang, Jun Wan, Wenwen Min

Abstract

While spatial transcriptomics (ST) has advanced our understanding of gene expression in tissue context, its high experimental cost limits its large-scale application. Predicting ST from pathology images is a promising, cost-effective alternative, but existing methods struggle to capture complex cross-slide spatial relationships. To address the challenge, we propose SpaHGC, a multi-modal heterogeneous graph-based model that captures both intra-slice and inter-slice spot-spot relationships from histology images. It integrates local spatial context within the target slide and cross-slide similarities computed from image embeddings extracted by a pathology foundation model. These embeddings enable inter-slice knowledge transfer, and SpaHGC further incorporates Masked Graph Contrastive Learning to enhance feature representation and transfer spatial gene expression knowledge from reference to target slides, enabling it to model complex spatial dependencies and significantly improve prediction accuracy. We conducted comprehensive benchmarking on seven matched histology-ST datasets from different platforms, tissues, and cancer subtypes. The results demonstrate that SpaHGC significantly outperforms the existing nine state-of-the-art methods across all evaluation metrics. Additionally, the predictions are significantly enriched in multiple cancer-related pathways, thereby highlighting its strong biological relevance and application potential.

Cross-Slice Knowledge Transfer via Masked Multi-Modal Heterogeneous Graph Contrastive Learning for Spatial Gene Expression Inference

Abstract

While spatial transcriptomics (ST) has advanced our understanding of gene expression in tissue context, its high experimental cost limits its large-scale application. Predicting ST from pathology images is a promising, cost-effective alternative, but existing methods struggle to capture complex cross-slide spatial relationships. To address the challenge, we propose SpaHGC, a multi-modal heterogeneous graph-based model that captures both intra-slice and inter-slice spot-spot relationships from histology images. It integrates local spatial context within the target slide and cross-slide similarities computed from image embeddings extracted by a pathology foundation model. These embeddings enable inter-slice knowledge transfer, and SpaHGC further incorporates Masked Graph Contrastive Learning to enhance feature representation and transfer spatial gene expression knowledge from reference to target slides, enabling it to model complex spatial dependencies and significantly improve prediction accuracy. We conducted comprehensive benchmarking on seven matched histology-ST datasets from different platforms, tissues, and cancer subtypes. The results demonstrate that SpaHGC significantly outperforms the existing nine state-of-the-art methods across all evaluation metrics. Additionally, the predictions are significantly enriched in multiple cancer-related pathways, thereby highlighting its strong biological relevance and application potential.
Paper Structure (31 sections, 30 equations, 15 figures, 10 tables)

This paper contains 31 sections, 30 equations, 15 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Problem Formulation
  5. Heterogeneous Graph Construction
  6. Complementary Masking Strategy
  7. Cross Node Dual Attention (CNDA) Module
  8. Cross Node Attention Pool (CNAP) Module
  9. Loss Function
  10. Experiments
  11. Experimental Setup
  12. Datasets.
  13. Baselines and Evaluation Metrics.
  14. Implementation Details.
  15. Preventing Data Leakage in Nested LOOCV.
  16. ...and 16 more sections

Figures (15)

  • Figure 1: Overview of SpaHGC. (A) The inputs, outputs, and objectives of SpaHGC. (B) Key Contribution of SpaHGC: Constructing a multi-modal heterogeneous graph from cross-slice spatial transcriptomics and histology data.
  • Figure 2: The overall architecture of SpaHGC. (A) Data preprocessing and construction of the multi-modal heterogeneous graph. (B) The backbone network of SpaHGC. (C) Architecture of the CNDA module. (D) Architecture of the CNAP module. (E) Downstream analysis.
  • Figure 3: Visualization of predicted expression patterns for marker genes. Results are shown for SpaHGC and nine baseline methods. (A) SPINK5 on the cSCC dataset. (B) CEACAM5 on the Pancreas1 dataset.
  • Figure S1: Per-slice PCC scores of predicted gene expression on the cSCC datasets.
  • Figure S2: Per-slice PCC scores of predicted gene expression across the Alex, Lymph Node, and Pancreas1 datasets.
  • ...and 10 more figures