Bridging Large Language Models and Single-Cell Transcriptomics in Dissecting Selective Motor Neuron Vulnerability

TL;DR

The paper tackles deciphering cell identity and selective vulnerability in neurodegenerative diseases from single-cell data by introducing a multimodal embedding that maps top expressed genes per cell to NCBI gene descriptions and encodes them with multiple LLMs, producing expression-weighted cell embeddings. These embeddings, computed as $\mathbf{z}_i = \frac{\sum_{j \in \mathcal{T}_i} x_{ij} \mathbf{g}_j}{\sum_{j in \mathcal{T}_i} x_{ij}}$, are then used in a KNN framework with cosine distance to classify motor neuron subtypes (e.g., $\gamma$ MNs and $\alpha$ MNs of types $\alpha$-FF, $\alpha$-FR, $\alpha$-SF). Among the evaluated models, OpenAI's text-embedding-ada-002 achieved the highest accuracy (~0.70) and balanced F1/kappa, with BioBERT and SciBERT providing competitive performance, while newer text-embedding-3 variants underperformed. The study highlights both the promise and limitations of task-specific, text-derived embeddings for resolving fine-grained transcriptomic heterogeneity and suggests avenues such as contrastive learning, gene-level attention, and spatial/temporal extensions to improve performance and interpretability in neurodegenerative contexts.

Abstract

Understanding cell identity and function through single-cell level sequencing data remains a key challenge in computational biology. We present a novel framework that leverages gene-specific textual annotations from the NCBI Gene database to generate biologically contextualized cell embeddings. For each cell in a single-cell RNA sequencing (scRNA-seq) dataset, we rank genes by expression level, retrieve their NCBI Gene descriptions, and transform these descriptions into vector embedding representations using large language models (LLMs). The models used include OpenAI text-embedding-ada-002, text-embedding-3-small, and text-embedding-3-large (Jan 2024), as well as domain-specific models BioBERT and SciBERT. Embeddings are computed via an expression-weighted average across the top N most highly expressed genes in each cell, providing a compact, semantically rich representation. This multimodal strategy bridges structured biological data with state-of-the-art language modeling, enabling more interpretable downstream applications such as cell-type clustering, cell vulnerability dissection, and trajectory inference.

Figures (12)

• Figure 1: Schematic of the embedding generation pipeline. Gene expression values from a scRNA-seq matrix are used to rank genes per cell. The top ranked genes are mapped to NCBI gene summaries, which are embedded using a language model. Expression-weighted embeddings are averaged to produce cell-level embeddings, which are then used for downstream tasks such as motor neuron vulnerability classification and subtype clustering.
• Figure 2: Accuracy, weighted F1, Cohen's Kappa scores for motor neuron subtype classification using different embedding models.
• Figure 3: Confusion matrix using text-embedding-ada-002 embeddings.
• Figure 4: Confusion matrix using text-embedding-3-small embeddings.
• Figure 5: Confusion matrix using text-embedding-3-large embeddings.