ChromFound: Towards A Universal Foundation Model for Single-Cell Chromatin Accessibility Data

TL;DR

ChromFound presents the first universal foundation model for single-cell ATAC-seq data by integrating genome-aware OCR tokenization with a hybrid encoder that jointly models local and long-range chromatin interactions. Pretrained on ~1.97 million cells across 30+ tissues and 6 diseases, it achieves robust zero-shot cell representations and strong transfer to cell-type annotation and cross-omics prediction, while revealing enhancer–gene links and perturbation responses in the noncoding genome. The architecture combines a Window Partition Self-Attention module for local context and a Mamba block for scalable long-range modeling, enabling genome-wide coverage with efficient computation. This work advances scalable, interpretable epigenomic modeling and multi-omics integration in single-cell biology, offering a foundation for mapping cis-regulatory landscapes at scale.

Abstract

The advent of single-cell Assay for Transposase-Accessible Chromatin using sequencing (scATAC-seq) offers an innovative perspective for deciphering regulatory mechanisms by assembling a vast repository of single-cell chromatin accessibility data. While foundation models have achieved significant success in single-cell transcriptomics, there is currently no foundation model for scATAC-seq that supports zero-shot high-quality cell identification and comprehensive multi-omics analysis simultaneously. Key challenges lie in the high dimensionality and sparsity of scATAC-seq data, as well as the lack of a standardized schema for representing open chromatin regions (OCRs). Here, we present ChromFound, a foundation model tailored for scATAC-seq. ChromFound utilizes a hybrid architecture and genome-aware tokenization to effectively capture genome-wide long contexts and regulatory signals from dynamic chromatin landscapes. Pretrained on 1.97 million cells from 30 tissues and 6 disease conditions, ChromFound demonstrates broad applicability across 6 diverse tasks. Notably, it achieves robust zero-shot performance in generating universal cell representations and exhibits excellent transferability in cell type annotation and cross-omics prediction. By uncovering enhancer-gene links undetected by existing computational methods, ChromFound offers a promising framework for understanding disease risk variants in the noncoding genome.

Figures (9)

• Figure 1: An overview of ChromFound architecture. The methods of ChromFound are listed in Section \ref{['sec:methods']}. OCR tokenization are described in Section \ref{['tokenization']}. The hybrid encoder layer is described in Section \ref{['sec:encoder']}. Other pretraining details of datasets and implementation are included in Section \ref{['sec:pretrainig']}. All evaluation results are detailed in Section \ref{['sec:experiments']}.
• Figure 2: Results of denoising low count. The left y-axis shows the absolute metric values (scatter plot) and the right y-axis indicates the relative gains (%) over the second-best methods (bar plot).
• Figure 3: Results of cell type annotation, evaluated by Accuracy (left) and macro F1 Score (right).
• Figure 4: (a) ROC curves illustrate the performance of enhancer-gene link prediction, where true positive rates are plotted against false positive rates across varying prediction thresholds. Ground truth labels are defined based on the significance (FDR < 0.05) of expression changes after enhancer perturbation. (b) Scatter plots represent a specific enhancer perturbation. The y-axis shows the predicted scores of average gene responses of enhancer perturbation, while the x-axis represents the real effects of post-perturbation. Both magnitudes are rescaled to the unit norm.
• Figure 5: The plots illustrate ChromFound's superior performance in denoising batch effect on Bone To326K to2024multi.