Cell-ontology guided transcriptome foundation model

TL;DR

This paper introduces scCello, a cell-ontology guided transcriptome foundation model that leverages a cell ontology graph to improve cell representations learned from large-scale scRNA-seq data. It integrates three training objectives—masked gene prediction ${L_{MGP}}$, intra-cellular coherence ${L_{Intra}}$, and inter-cellular relational alignment ${L_{Inter}}$—alongside a regularization term ${L_{Reg}}$, all optimized together to produce biologically meaningful embeddings. By grounding representations in ontology-derived similarities via Personalized PageRank, scCello achieves strong zero-shot and transfer performance across diverse downstream tasks, including identifying unseen cell types, predicting marker genes, and forecasting cancer drug responses, while also showing robustness to batch effects. The results demonstrate substantial gains over ontology-agnostic TFMs and highlight the value of incorporating structured biological knowledge into pre-training, with a notably smaller parameter footprint than comparable models. Limitations include the need for continual ontology updates and scaling, but the approach offers a practical, general-purpose foundation for rapid cellular discovery and personalized medicine applications.

Abstract

Transcriptome foundation models TFMs hold great promises of deciphering the transcriptomic language that dictate diverse cell functions by self-supervised learning on large-scale single-cell gene expression data, and ultimately unraveling the complex mechanisms of human diseases. However, current TFMs treat cells as independent samples and ignore the taxonomic relationships between cell types, which are available in cell ontology graphs. We argue that effectively leveraging this ontology information during the TFM pre-training can improve learning biologically meaningful gene co-expression patterns while preserving TFM as a general purpose foundation model for downstream zero-shot and fine-tuning tasks. To this end, we present single cell, Cell-ontology guided TFM scCello. We introduce cell-type coherence loss and ontology alignment loss, which are minimized along with the masked gene expression prediction loss during the pre-training. The novel loss component guide scCello to learn the cell-type-specific representation and the structural relation between cell types from the cell ontology graph, respectively. We pre-trained scCello on 22 million cells from CellxGene database leveraging their cell-type labels mapped to the cell ontology graph from Open Biological and Biomedical Ontology Foundry. Our TFM demonstrates competitive generalization and transferability performance over the existing TFMs on biologically important tasks including identifying novel cell types of unseen cells, prediction of cell-type-specific marker genes, and cancer drug responses.

Figures (12)

• Figure 1: (a) Cell ontology graph describes taxonomic relationships between cell types. (b) Each cell in scRNA-seq data is represented by gene sequences, and associated with a cell type ontology identifier. (c) The pre-training framework of scCello is structured with three levels of objectives: gene-level masked gene prediction, intra-cellular level cell type coherence and inter-cellular level ontology alignment. For example, as shown in panel b, cells 1, 2, and 3 are labelled with cell type A, B and C. The intra-cellular cell type coherence loss encourages alignment of embedding $\mathbf{z}_1$ with $\mathbf{h}_A$, $\mathbf{z}_2$ with $\mathbf{h}_B$, and $\mathbf{z}_3$ with $\mathbf{h}_C$. The inter-cellular level ontology alignment loss encourages representational learning of cell similarities $\mathbf{z}^\top_i\mathbf{z}_j$ between cell $i$ and $j$ to be consistent to the similarity of their corresponding cell types $sim(c_i, c_j)$ based on the ontology relationships. (d) Downstream tasks enabled by scCello and demonstrated in the study.
• Figure 2: Novel cell type classification on OOD cell type dataset $D_{1}^{ct}$ for increasing difficulties.
• Figure 3: Batch integration on the curated ID and OOD datasets.
• Figure 4: Graphical illustration of applying the Personalized PageRank (PPR) algorithm to cell ontology graph. As explained in App. \ref{['app:ppr_transformation']}, PPR conducts random walks over the ontology graph with respect to a target cell type $u$, and converges to a steady state when the likelihood of terminating on each node stabilizes into a steady distribution. This likelihood distribution determines the final PPR score $\mathrm{PPR}(\cdot)$ and reflects the structural similarity between cell types.
• Figure 5: Comparison of the distributions for the PPR scores $\mathrm{PPR}(\cdot)$ and the structural similarity $\mathrm{sim}(\cdot)$ after the transformation.