TFBS-Finder: Deep Learning-based Model with DNABERT and Convolutional Networks to Predict Transcription Factor Binding Sites

TL;DR

TFBS-Finder introduces a DNABERT-based framework that combines CNN-derived local features with attention-enhanced multi-scale context to predict Transcription Factor Binding Sites. By training and evaluating on 165 ENCODE ChIP-seq datasets, the model demonstrates superior accuracy, PR-AUC, and ROC-AUC compared with multiple state-of-the-art predictors, aided by comprehensive ablation and cross-cell line validations. The architecture integrates DNABERT embeddings with a CNN module, Modified Convolutional Block Attention Module (MCBAM), and Multi-Scale Convolution with Attention (MSCA), unified in an output module for robust TFBS discrimination. The results suggest strong generalization across cell types and TFs, with public availability of code and data, highlighting practical utility for gene regulatory network analyses and disease variant interpretation.

Abstract

Transcription factors are proteins that regulate the expression of genes by binding to specific genomic regions known as Transcription Factor Binding Sites (TFBSs), typically located in the promoter regions of those genes. Accurate prediction of these binding sites is essential for understanding the complex gene regulatory networks underlying various cellular functions. In this regard, many deep learning models have been developed for such prediction, but there is still scope of improvement. In this work, we have developed a deep learning model which uses pre-trained DNABERT, a Convolutional Neural Network (CNN) module, a Modified Convolutional Block Attention Module (MCBAM), a Multi-Scale Convolutions with Attention (MSCA) module and an output module. The pre-trained DNABERT is used for sequence embedding, thereby capturing the long-term dependencies in the DNA sequences while the CNN, MCBAM and MSCA modules are useful in extracting higher-order local features. TFBS-Finder is trained and tested on 165 ENCODE ChIP-seq datasets. We have also performed ablation studies as well as cross-cell line validations and comparisons with other models. The experimental results show the superiority of the proposed method in predicting TFBSs compared to the existing methodologies. The codes and the relevant datasets are publicly available at https://github.com/NimishaGhosh/TFBS-Finder/.

Figures (6)

• Figure 1: Pipeline of the Work
• Figure 2: Two submodules of MCBAM module where (a) represents the spatial attention block and (b) represents the channel attention block
• Figure 3: Prediction performance of TFBS-Finder compared with other variants where (a), (b) and (c) represents accuracy, PR-AUC and ROC-AUC. The white line inside the violins represent the median while the bold black lines show the interquartile range. The two vertical thin lines inside the violins show the range of maximum and minimum non-outlier values and the very large thin lines represent the outliers.
• Figure 4: Venn Diagram to show the common TFs between (a) Gm12878 and Helas3, (b) Gm12878 and Hepg2, (c) Gm12878 and K562, (d) Hepg2 and Helas3, (e) Hepg2 and K562 and (f) Helas3 and K562
• Figure 5: Prediction performance of TFBS-Finder for cross-cell line and traditional validations where (a), (b), (c) and (d) show the results for training on Gm12878, Helas3, Hepg2 and K562. The line inside the boxes represent the median while the top and bottom edges show the upper and lower quartiles respectively. The circles represent the outlier values.