Diffusion Generative Modeling for Spatially Resolved Gene Expression Inference from Histology Images

TL;DR

This work addresses the challenge of inferring spatially resolved gene expression from histology images, a difficult one-to-many problem due to biological heterogeneity and measurement noise. It introduces Stem, a conditional diffusion model that learns the distribution $p_{ ext{gene}}(X|V)$ using histology embeddings from pathology foundation models to condition a Diffusion Transformer on image information. The approach achieves state-of-the-art results across multiple datasets (Kidney Visium, HER2ST, and beyond) while preserving biological variability as measured by a new metric, Relative Variation Distance (RVD), and enables downstream tissue-structure annotations from predicted gene profiles. By enabling in silico genomics directly from readily available H&E images, Stem reduces reliance on expensive ST experiments and offers a robust, scalable tool for discovery in cancer and healthy tissues.

Abstract

Spatial Transcriptomics (ST) allows a high-resolution measurement of RNA sequence abundance by systematically connecting cell morphology depicted in Hematoxylin and Eosin (H&E) stained histology images to spatially resolved gene expressions. ST is a time-consuming, expensive yet powerful experimental technique that provides new opportunities to understand cancer mechanisms at a fine-grained molecular level, which is critical for uncovering new approaches for disease diagnosis and treatments. Here, we present $\textbf{Stem}$ ($\textbf{S}$pa$\textbf{T}$ially resolved gene $\textbf{E}$xpression inference with diffusion $\textbf{M}$odel), a novel computational tool that leverages a conditional diffusion generative model to enable in silico gene expression inference from H&E stained images. Through better capturing the inherent stochasticity and heterogeneity in ST data, $\textbf{Stem}$ achieves state-of-the-art performance on spatial gene expression prediction and generates biologically meaningful gene profiles for new H&E stained images at test time. We evaluate the proposed algorithm on datasets with various tissue sources and sequencing platforms, where it demonstrates clear improvement over existing approaches. $\textbf{Stem}$ generates high-fidelity gene expression predictions that share similar gene variation levels as ground truth data, suggesting that our method preserves the underlying biological heterogeneity. Our proposed pipeline opens up the possibility of analyzing existing, easily accessible H&E stained histology images from a genomics point of view without physically performing gene expression profiling and empowers potential biological discovery from H&E stained histology images.

Figures (16)

• Figure 1: Overview of Stem. The input training data for Stem is ST datasets that contain both H&E images and spot-wise gene expression profiles. During training, gene counts and gene types are separately embedded and combined to serve as the input into DiT blocks. Images are cropped into $224 \times 224$ patches surrounding every spot and then tokenized via pathology foundation models. Fused image tokens serve as the conditions and are input into every DiT block. After training, gene expression output could be iteratively sampled conditioned on any input image patch.
• Figure 2: Visualization of neural network architecture in Stem. Histology Images are embedded into tokens with pathology foundation models and then pooled into condition hidden vectors in Stem. Count values for each input gene is first scaled up by the gene count encoder and then combined with a trainable gene type embedding matrix. The backbone of Stem follows the design of DiT blocks and training scheme for Stem follows DDPM (see Sec \ref{['sec:method']} for more details).
• Figure 3: Visualization of unsupervised clustering results and cancer biomarker genes.
• Figure 4: Gene variation comparison between prediction and ground truth for HMHVGs in the Kidney Visium dataset. A closer match to the blue curve is better.
• Figure 5: Gene variation comparison between prediction and ground truth for HVGs in the Kidney Visium dataset. A closer match to the blue curve is better.