MERGE: Multi-faceted Hierarchical Graph-based GNN for Gene Expression Prediction from Whole Slide Histopathology Images

TL;DR

MERGE addresses spatial gene-expression prediction from whole-slide histopathology by constructing a multi-faceted hierarchical graph that captures both local morphology and long-range interactions among tissue patches. It combines spatial and feature-space clustering to form intra-cluster edges and provides centroid-based shortcut edges to enable efficient information flow via a Graph Attention Network, while employing a ResNet18-based patch encoder and a gene-aware smoothing technique (SPCS). The approach yields superior predictive performance across datasets, outperforming state-of-the-art baselines on metrics like $PCC$, $MSE$, and $MAE$, and is supported by qualitative heatmaps and ablation analyses that validate the design choices. Together, MERGE advances robust, morphology-guided gene-expression prediction from WSIs with practical implications for leveraging histology to infer spatial transcriptomics profiles in clinical and research settings.

Abstract

Recent advances in Spatial Transcriptomics (ST) pair histology images with spatially resolved gene expression profiles, enabling predictions of gene expression across different tissue locations based on image patches. This opens up new possibilities for enhancing whole slide image (WSI) prediction tasks with localized gene expression. However, existing methods fail to fully leverage the interactions between different tissue locations, which are crucial for accurate joint prediction. To address this, we introduce MERGE (Multi-faceted hiErarchical gRaph for Gene Expressions), which combines a multi-faceted hierarchical graph construction strategy with graph neural networks (GNN) to improve gene expression predictions from WSIs. By clustering tissue image patches based on both spatial and morphological features, and incorporating intra- and inter-cluster edges, our approach fosters interactions between distant tissue locations during GNN learning. As an additional contribution, we evaluate different data smoothing techniques that are necessary to mitigate artifacts in ST data, often caused by technical imperfections. We advocate for adopting gene-aware smoothing methods that are more biologically justified. Experimental results on gene expression prediction show that our GNN method outperforms state-of-the-art techniques across multiple metrics.

Figures (19)

• Figure 1: Combining image feature-space clustering with spatial clustering to build a multi-faceted hierarchical graph leads to spatial gene expression prediction. Left: Patches extracted from a WSI. Middle: Edges within and across clusters allows us to model short and long range interactions among patches. Right: This allows our GNN model to make robust predictions.
• Figure 2: The schematic of MERGE shows the overall workflow of our method. (a) Outlines the architecture of our method. The ResNetSimCLR model is fine-tuned on the gene expression prediction task using MSE loss. The last layer is discarded to yield 256-dimensional feature vectors for the patches. The graph construction step produces the multi-faceted hierarchical graph for our GNN, which is trained on MSE loss. The output of the GNN is a 250-dimensional gene expression vector at each node. (b) Shows the graph construction strategy demonstrated through reduced examples. The left column shows feature space clustering and the right column shows spatial clustering. The internal edges of a cluster are shown in white, while the shortcut edge is shown in blue. The two yellow spots represent the centroid spots of the two clusters.
• Figure 3: The two columns show the original and smoothed expressions for two cancer-relevant genes (FASN and GNAS) in a tissue sample. The X-axis represents bins of gene expression values, while the Y-axis shows the number of spots belonging to those bins. It is evident that 8n smoothing drastically reduces the scale of the expression values. While SPCS also reduces the scale of the data, it does so being transcriptome-aware.
• Figure 4: From left to right the four figures in each row represent the original gene expressions, SPCS smoothed gene expressions, 8n smoothed gene expressions, and the WSI. We can see that for both genes and both samples, SPCS outputs have clear correspondence with tissue morphology.
• Figure 5: SPCS improves gene expression correspondence with histology labels. Each panel shows the adjusted rand index (ARI) of K-Medoids clusters using various gene expression matrices. Comparison is directly taken from the original SPCS 10.1093/bib/bbac116 paper.