What makes for good morphology representations for spatial omics?

TL;DR

The paper addresses how to design morphology representations that enhance spatial omics analyses by organizing methods into translation—predicting gene expression from morphology—and integration—augmenting spatial domain discovery with morphological cues. It formalizes a framework using paired modalities $(x_G, x_M, y)$ and encoders that yield $h_G$ and $h_M$, with mutual information $I(h_G; h_M)$ and $I(h_G; y)$ guiding four qualitative regimes. It surveys morphology-translation methods (gene expression prediction, multi-scale and bi-modal architectures) and morphology-integration approaches (domain identification with fusion strategies), datasets, evaluation metrics, and practical challenges. The authors provide a roadmap to develop task-relevant, complementary morphological descriptors, improve benchmarking, and extend the framework to broader multi-modal data, aiming to improve robustness, reproducibility, and clinical relevance in spatial omics analyses.

Abstract

Spatial omics has transformed our understanding of tissue architecture by preserving spatial context of gene expression patterns. Simultaneously, advances in imaging AI have enabled extraction of morphological features describing the tissue. The intersection of spatial omics and imaging AI presents opportunities for a more holistic understanding. In this review we introduce a framework for categorizing spatial omics-morphology combination methods, focusing on how morphological features can be translated or integrated into spatial omics analyses. By translation we mean finding morphological features that spatially correlate with gene expression patterns with the purpose of predicting gene expression. Such features can be used to generate super-resolution gene expression maps or infer genetic information from clinical H&E-stained samples. By integration we mean finding morphological features that spatially complement gene expression patterns with the purpose of enriching information. Such features can be used to define spatial domains, especially where gene expression has preceded morphological changes and where morphology remains after gene expression. We discuss learning strategies and directions for further development of the field.

Figures (4)

• Figure 1: (a) Feature extraction from spatial omics and morphology on 10X Visium spatial transcriptomics and H&E erickson2022spatially. We use the $n$th spot in Visium $x_{G}^n$ as the center to extract the $n$th image patch $x_{M}^n$. The modality specific encoders $e_{\theta_{G}}$ and $e_{\theta_M}$ will output the molecular feature vector $h_{G}^n$ and the morphological feature vector $h_{M}^n$ for position $(i^n, j^n)$. (b) Intuition of the framework. By assuming the amount of relevant spatial omics information is fixed (solid blue area), we get four different scenarios depending on the morphological features. We express these four scenarios by the quadrants formed when presenting the relevance of the morphological features as y-axis and their shared information with the spatial omics features as x-axis.
• Figure 2: The most common task involving morphology translation is the prediction of gene expression. This can be done either by inferring the gene expression in a new sample or by imputing the gene expression in between areas containing gene expression.
• Figure 3: The most common task involving morphology integration is the identification of spatial domains which can be then used in downstream tasks.
• Figure 4: Commonly used metrics for morphology translation and integration. This synthetic example presents tissue regions as concentric circles. (a) Translation metrics usually measure the agreement of the true gene expression with the gene expression predicted from morphology. Pearson's correlation coefficient (PCC) and regression metrics such as the root mean squared error (RMSE) are usually employed. (b) Integration metrics measure the agreement of expert annotations with the domains defined jointly morphology and spatial transcriptomics via the adjusted Rand index (ARI). It is common to also define spatially variable genes that define the identified domains and measure their degree of spatial auto-correlation with Moran's I or Geary's C.