A New Framework for Explainable Rare Cell Identification in Single-Cell Transcriptomics Data

TL;DR

This work tackles the lack of gene-level explanations for rare cells detected in single-cell transcriptomics by replacing PCA-based pipelines with a PCA-free framework that operates on Highly Variable Genes ($HVGs$). It integrates the Isolation Distributional Kernel ($IDK$) for robust, high-dimensional anomaly scoring and the improved SiNNe explainer to produce concise, gene-subspace explanations for each anomaly and for representative normal cells, enabling both instance- and cluster-level interpretability. The authors demonstrate that HVG-based detection preserves subtle signals better than PCA, validate explanations across tutorials and complex real datasets, and show that different detectors identify complementary anomalies, justifying multi-detector approaches. They also introduce rigorous validation of anomaly groups using $IDK^2$, and extend explanations to cross-type and large-diversity datasets, highlighting practical impact for explainable rare-cell biology and broader high-dimensional data analysis. The framework thus provides verifiable, gene-level insights into why a cell is rare, facilitating biological interpretation and methodological verification in single-cell studies.

Abstract

The detection of rare cell types in single-cell transcriptomics data is crucial for elucidating disease pathogenesis and tissue development dynamics. However, a critical gap that persists in current methods is their inability to provide an explanation based on genes for each cell they have detected as rare. We identify three primary sources of this deficiency. First, the anomaly detectors often function as "black boxes", designed to detect anomalies but unable to explain why a cell is anomalous. Second, the standard analytical framework hinders interpretability by relying on dimensionality reduction techniques, such as Principal Component Analysis (PCA), which transform meaningful gene expression data into abstract, uninterpretable features. Finally, existing explanation algorithms cannot be readily applied to this domain, as single-cell data is characterized by high dimensionality, noise, and substantial sparsity. To overcome these limitations, we introduce a framework for explainable anomaly detection in single-cell transcriptomics data which not only identifies individual anomalies, but also provides a visual explanation based on genes that makes an instance anomalous. This framework has two key ingredients that are not existed in current methods applied in this domain. First, it eliminates the PCA step which is deemed to be an essential component in previous studies. Second, it employs the state-of-art anomaly detector and explainer as the efficient and effective means to find each rare cell and the relevant gene subspace in order to provide explanations for each rare cell as well as the typical normal cell associated with the rare cell's closest normal cells.

Figures (9)

• Figure 1: Overview of existing methods Grün2015Single-cellZhou2025AnomalGRNFerré2021Anomaly and our proposed framework. In the proposed framework, we first select highly variable genes and compute anomaly scores using the Isolation Distributional Kernel (IDK) Xu2020IDK, which measures each cell’s similarity to the dataset distribution. We then rank cells and identify the top anomaly and typical normal. Finally, we apply SiNNe samariya2020new to extract concise gene subspaces explaining why a specific cell is anomalous or typically normal.
• Figure 2: Visualization of SiNNe‐derived feature subspaces in the Tutorial dataset. Anomalous cells are shown as "x" in subfigures (a-c); and the typical normal instance $\textbf{z}^{293T}$ is shown as " " in subfigure (d), which is compared against all cells in $\mathcal{C}_{293T}$ (blue dots) and the three detected anomalies.
• Figure 3: Global similarity score distribution and spatial mapping of all instances on the t-SNE plot, with zero-score instances highlighted.
• Figure 4: SiNNe-derived explanations for typical anomalies sampled from $\mathcal{A}_0$. Each subfigure demonstrates a unique gene subspace that distinguishes the anomaly ("x") from the main 293T cell population (blue dots), confirming the heterogeneity of the zero-score group.
• Figure 5: Dual-reference explanation for the cross-type anomaly $\textbf{a}^{(1)}$ in the Airway dataset. The visualizations show its clear separation from (a) its nearest "Club" neighbors, (b) its own "Basal" population, and (c) both populations simultaneously.

Theorems & Definitions (3)

• Definition 1: Subspace Anomaly Group
• Definition 2: Isolation kernel ting2018isolationqin2019nearest, IK
• Definition 3: Isolation Distributional Kernel ting2023isolation