Multimodal Spatial Omics: From Data Acquisition to Computational Integration

TL;DR

This article surveys the rapidly growing field of multimodal spatial omics, detailing two primary acquisition strategies—adjacent-slice and simultaneous co-profiling—across sequencing and imaging modalities. It then categorizes computational integration approaches into probabilistic inference, matrix factorization, optimal transport, and deep learning/foundation-model frameworks, linking each to core tasks like co-registration, deconvolution, and image-to-omics prediction. The review highlights representative methods (e.g., Stereoscope, MISAR-seq, PASTE, COSMOS, SPATIA, SpaLLM, OmiCLIP) and discusses their strengths, limitations, and applicable contexts, while stressing the need for standardization, benchmarking, and scalable infrastructure. Collectively, the work emphasizes that advances in experimental design, algorithmic development, and community resources will enable robust, interpretable, and clinically actionable insights from the spatial organization of multi-omics data.

Abstract

Recent developments in spatial omics technologies have enabled the generation of high dimensional molecular data, such as transcriptomes, proteomes, and epigenomes, within their spatial tissue context, either through coprofiling on the same slice or through serial tissue sections. These datasets, which are often complemented by images, have given rise to multimodal frameworks that capture both the cellular and architectural complexity of tissues across multiple molecular layers. Integration in such multimodal data poses significant computational challenges due to differences in scale, resolution, and data modality. In this review, we present a comprehensive overview of computational methods developed to integrate multimodal spatial omics and imaging datasets. We highlight key algorithmic principles underlying these methods, ranging from probabilistic to the latest deep learning approaches.

Figures (5)

• Figure 1: Experimental workflows for spatial multi-omics data acquisition. The figure illustrates three major experimental approaches for generating multimodal spatial omics data. A: Serial sectioning workflow where consecutive tissue sections are processed using different modalities. Following tissue sectioning, individual slices undergo staining and are analyzed using distinct spatial omics technologies (e.g., spatial transcriptomics on one section, spatial proteomics on another). Subsequent computational co-registration aligns the datasets to a common coordinate system, enabling cross-modality integration of spatially resolved molecular profiles. B: Measuring several molecular types at once on the same piece of tissue. Once the tissue has been made ready and stained, both RNA and protein (or other types) are taken from the same places in space, which removes the need for computer-based matching and keeps exact location links between measurements. C: Comparison of sequencing-based and imaging-based spatial omics platforms. The NGS-based workflow (left) shows tissue sectioning, spatial barcoding via array or bead-based capture, cDNA synthesis and library preparation, followed by next-generation sequencing and computational analysis to generate integrated spatial multi-omics data. The image-based workflow (right) depicts in situ detection methods using sequential rounds of hybridization or antibody labeling, high-resolution imaging, and image analysis to produce spatially resolved molecular profiles with subcellular resolution.
• Figure 2: Fusion strategies for multimodal spatial omics integration. Three principal approaches for combining information from multiple spatial omics modalities. Early Fusion: Raw features from different modalities (e.g., spatial transcriptomics, proteomics, histology) are concatenated into a single input matrix before being processed by a unified machine learning model. Intermediate Fusion: Each modality is first processed independently to extract modality-specific latent representations, which are then concatenated at the feature level and fed into a downstream model. This approach allows modality-specific preprocessing while enabling the model to learn cross-modal interactions. Late Fusion: Separate models process each modality, returning independent predictions that are later aggregated (e.g., via voting, averaging, or weighted combination) to generate the final output. The bottom panel illustrates key downstream computational tasks enabled by these integration strategies: co-registration of modalities, cell-type deconvolution, spatial domain identification and clustering, prediction of molecular profiles from H&E images, and generative modeling for data imputation and augmentation.
• Figure 3: Landscape of computational tools for multimodal spatial omics integration. Overview of published computational methods mapped by the modality combinations they integrate and the downstream analytical tasks they address. Methods are categorized by five key computational tasks: co-registration of modalities, cell-type deconvolution, spatial domain identification, prediction of molecular profiles from H&E images, and generative modeling. The timeline illustrates the rapid expansion of the field, with a marked acceleration in method development from 2022 onwards. Notably, the integration of spatial transcriptomics with H&E images and scRNA-seq references represents the most mature area, with numerous established tools available. In contrast, computational strategies for integrating of spatial epigenomics, spatial metabolomics, and multi-modal spatial proteomics remain comparatively underexplored, highlighting key opportunities for future methodological development. Note: some tools support multiple computational tasks; for ordering and row shading we show the primary task.
• Figure 4: Computational frameworks for multimodal spatial omics integration.A: Probabilistic and statistical inference methods. Multimodal spatial omics data are modeled using shared latent biological variables $z_n$ associated with each spatial location $n$. These latent variables capture underlying biological structure such as cell types, cell states, or molecular programs. Each modality is generated from $z_n$ through a modality-specific likelihood $p(X^{(m)} \mid z_n, \Theta^{(m)})$, while prior distributions $p(z_n \mid \Phi)$ encode biological or structural assumptions. Bayesian or likelihood-based inference is used to estimate posterior distributions $p(z_n \mid X^{(1)}, X^{(2)}, \ldots)$, enabling uncertainty-aware multimodal integration. B: Matrix factorization and latent variable models. High-dimensional data matrices from multiple modalities are decomposed into shared low-dimensional latent factors $Z \in \mathbb{R}^{N \times K}$ and modality-specific loading matrices $W^{(m)} \in \mathbb{R}^{K \times G_m}$, such that $X^{(m)} \approx Z W^{(m)}$. The shared latent factors capture common biological structure across modalities and support interpretable downstream analysis, including identification of cell types, gene programs, and spatial or functional patterns. C: Wasserstein optimal transport (OT) quantifies the minimal cost to transform one distribution $\mu$ into another $\nu$ through a transport map $T$, where the cost is defined by a function $C(x,y)$ measuring the dissimilarity when matching samples from $\mu$ and $\nu$, enabling alignment of datasets in a common feature space. D: Gromov-Wasserstein OT aligns datasets across different feature spaces by preserving pairwise distance relationships within each modality, matching points based on structural similarity rather than direct feature correspondence, with the goal of finding a map $T$ that minimizes $\left| d_X - d_Y \right|$. E: Fused Gromov-Wasserstein OT combines feature-level costs (Wasserstein term) with structure-preserving costs (Gromov term), weighted by parameter $\alpha$, enabling simultaneous alignment of expression similarities and spatial or morphological distances.
• Figure 5: Deep learning architectures of multimodal spatial omics analysis.A: CNNs process histology image patches through successive convolutional layers that extract hierarchical features, from low-level textures to high-level tissue structures. Pooling operations reduce spatial dimensionality, and fully connected layers produce task-specific outputs such as gene expression predictions. B: Graph-based neural networks represent tissue locations or cells as nodes with edges encoding spatial adjacency or molecular similarity. In GCNs, node features are updated by aggregating neighbor information weighted by fixed degree-normalized coefficients (c$\textbackslash{}_ij$). GATs expand this by realizing attention weights that adaptively modulate neighbor contributions, enabling the model to focus on the most informative spatial or molecular associations. C:Transformers use self-attention to capture dependencies over large distances. In spatial omics, genes or proteins can be treated as input sequences, andpositional encodings can maintain spatial information. Multi-head attention computes pairwise relationships across the sequence, enabling discovery of coordinated gene programs. ViTs adjust this framework for histology images by partitioning images into patches, embedding each as a token, and applying transformer layers to capture global morphological context. D: Autoencoders learn compact representation of high-dimensional spatial omics data by encoding the inputs into a low-dimensional bottleneck and reconstruct them back through a decoder. The reconstruction loss encourages the latent space to preserve biologically meaningful structure, making autoencoders useful for multi-spatial omic data analysis, data integration, and gene imputation.