scCluBench: Comprehensive Benchmarking of Clustering Algorithms for Single-Cell RNA Sequencing

TL;DR

scCluBench addresses fragmentation in scRNA-seq clustering benchmarks by providing a standardized framework with 36 diverse datasets and coverage of traditional, deep learning, graph-based, and foundation-model methods. It couples unified preprocessing and evaluation with downstream biological interpretation pipelines, including marker gene identification and cell type annotation via best-mapping and marker-overlap strategies. Key findings show traditional methods falter on large-scale data, graph-based methods grapple with over-smoothing, and current biological foundation models underperform clustering despite classification strengths, highlighting a gap between general representations and clustering objectives. The framework enables fair method comparisons, reveals robustness boundaries, and offers practical guidance for method selection and future methodological development in single-cell analysis.

Abstract

Cell clustering is crucial for uncovering cellular heterogeneity in single-cell RNA sequencing (scRNA-seq) data by identifying cell types and marker genes. Despite its importance, benchmarks for scRNA-seq clustering methods remain fragmented, often lacking standardized protocols and failing to incorporate recent advances in artificial intelligence. To fill these gaps, we present scCluBench, a comprehensive benchmark of clustering algorithms for scRNA-seq data. First, scCluBench provides 36 scRNA-seq datasets collected from diverse public sources, covering multiple tissues, which are uniformly processed and standardized to ensure consistency for systematic evaluation and downstream analyses. To evaluate performance, we collect and reproduce a range of scRNA-seq clustering methods, including traditional, deep learning-based, graph-based, and biological foundation models. We comprehensively evaluate each method both quantitatively and qualitatively, using core performance metrics as well as visualization analyses. Furthermore, we construct representative downstream biological tasks, such as marker gene identification and cell type annotation, to further assess the practical utility. scCluBench then investigates the performance differences and applicability boundaries of various clustering models across diverse analytical tasks, systematically assessing their robustness and scalability in real-world scenarios. Overall, scCluBench offers a standardized and user-friendly benchmark for scRNA-seq clustering, with curated datasets, unified evaluation protocols, and transparent analyses, facilitating informed method selection and providing valuable insights into model generalizability and application scope.

Figures (10)

• Figure 1: Overview of scCluBench: resources, evaluation protocols, and biological interpretation.
• Figure 2: Dataset distributions. (a) to (d) show dataset distributions by cell count, gene number, clusters, and sparsity, while (e) displays the distribution of samples between human and mouse datasets.
• Figure 3: A digest of the visualization of all baselines.
• Figure 4: A digest of representation similarity of all baselines.
• Figure 5: All biological analysis of scCDCG on Mauro Human Pancreas cells.