TransST: Transfer Learning Embedded Spatial Factor Modeling of Spatial Transcriptomics Data

TL;DR

TransST introduces a transfer-learning embedded spatial factor modeling framework to enhance spatial transcriptomics analysis by leveraging external labeled data. It combines supervised probabilistic dimension reduction on a source dataset, adaptive transfer of the learned loading matrix to a target dataset, and a spatial Gaussian mixture model with Markov random field smoothing to cluster cells with spatial coherence. Across simulations and real datasets (breast cancer, DLPFC brain, mouse embryo, cSCC), TransST improves clustering accuracy and identifies biologically meaningful cell types and driving genes, outperforming existing methods. The approach offers robust, scalable cross-study integration to better characterize cellular heterogeneity in spatial contexts and to detect biomarkers within spatially resolved data.

Abstract

Background: Spatial transcriptomics have emerged as a powerful tool in biomedical research because of its ability to capture both the spatial contexts and abundance of the complete RNA transcript profile in organs of interest. However, limitations of the technology such as the relatively low resolution and comparatively insufficient sequencing depth make it difficult to reliably extract real biological signals from these data. To alleviate this challenge, we propose a novel transfer learning framework, referred to as TransST, to adaptively leverage the cell-labeled information from external sources in inferring cell-level heterogeneity of a target spatial transcriptomics data. Results: Applications in several real studies as well as a number of simulation settings show that our approach significantly improves existing techniques. For example, in the breast cancer study, TransST successfully identifies five biologically meaningful cell clusters, including the two subgroups of cancer in situ and invasive cancer; in addition, only TransST is able to separate the adipose tissues from the connective issues among all the studied methods. Conclusions: In summary, the proposed method TransST is both effective and robust in identifying cell subclusters and detecting corresponding driving biomarkers in spatial transcriptomics data.

Figures (7)

• Figure 1: Overview of TransST. TransST involves three steps. Step 1 learns the weight matrix from the source data; Step 2 adaptively updates the weight matrix of the target data by leveraging information transferred from the source data; Step 3 models the low-dimensional representation for downstream analysis, such as cluster analysis and identification of differentially expressed genes.
• Figure 2: Comparison of TransST with multiple methods across various simulation settings. Black dots represent the averaged values. A: Comparison of the clustering performances among various methods when $\beta=0.5$ (solid line) and $\beta=1$ (dotted line), using the true number of clusters; B: Transferring power of TransST with different levels of noise in the target data; C: Heatmap of differentially expressed genes for each cell type identified by TransST with $\beta=1$; D: Visualization the distribution of gene expression across different cell clusters by ridge plot; E: Comparison of the clustering performances among various methods that can estimate the number of clusters; F: Absolute error $|\hat{K}-K|$ of estimating the number of clusters by various methods.
• Figure 3: Computational time (in seconds) of various methods in different settings.
• Figure 4: TranST enables accurate spatial mapping of scRNA-seq data in human HER2-positive tumors. A: Comparison of the clustering performance among various methods, using the true number of clusters; B: Comparison of methods that can estimate the number of clusters; C: Heatmap of the top 5 differentially expressed genes for each cell type in Sample H1, identified by TransST. Each cluster is annotated based on its association with morphological regions; D: Spatial heatmaps with estimated labels by various methods for Sample H1. Morphological regions in the Image are annotated by a pathologist into six distinct categories: adipose tissue (cyan), breast glands (green), cancer in situ (orange), connective tissue (blue), immune infiltrate (yellow), and invasive cancer (red).
• Figure 5: TransST enables accurate identification of brain layers of the DLPFC dataset. A: Comparison of the clustering performance among various methods, using the true number of clusters; B: Comparison of methods that can estimate the number of clusters; C: Spatial heatmaps with estimated labels by various methods for Sample 151669 ($K$ is unknown); D: UMAP plots for TransST with colors and shapes showing the sample IDs.