Tool Choice Matters: Evaluating edgeR vs. DESeq2 for Sensitivity, Robustness, and Cross-Study Performance

TL;DR

Overall, the findings highlight that while DESeq2 may identify more DEGs even under stringent significance conditions, edgeR yields more robust and generalizable gene sets for downstream classification and cross-study replication, which underscores key trade-offs in tool selection for transcriptomic analyses.

Abstract

Differential gene expression (DGE) analysis is foundational to transcriptomic research, yet tool selection can substantially influence results. This study presents a comprehensive comparison of two widely used DGE tools, edgeR and DESeq2, using real and semi-simulated bulk RNA-Seq datasets spanning viral, bacterial, and fibrotic conditions. We evaluated tool performance across three key dimensions: (1) sensitivity to sample size and robustness to outliers; (2) classification performance of uniquely identified gene sets within the discovery dataset; and (3) generalizability of tool-specific gene sets across independent studies. First, both tools showed similar responses to simulated outliers, with Jaccard similarity between the DEG sets from perturbed and original (unperturbed) data decreasing as more outliers were added. Second, classification models trained on tool-specific genes showed that edgeR achieved higher F1 scores in 9 of 13 contrasts and more frequently reached perfect or near-perfect precision. Dolan-More performance profiles further indicated that edgeR maintained performance closer to optimal across a greater proportion of datasets. Third, in cross-study validation using four independent SARS-CoV-2 datasets, gene sets uniquely identified by edgeR yielded higher AUC, precision, and recall in classifying samples from held-out datasets. This pattern was consistent across folds, with some test cases achieving perfect separation using edgeR-specific genes. In contrast, DESeq2-specific genes showed lower and more variable performance across studies. Overall, our findings highlight that while DESeq2 may identify more DEGs even under stringent significance conditions, edgeR yields more robust and generalizable gene sets for downstream classification and cross-study replication, which underscores key trade-offs in tool selection for transcriptomic analyses.

Figures (4)

• Figure 1: Sensitivity of edgeR and DESeq2 to sample size and outliers. Panels (a–d) show DEG counts and top significant genes for edgeR across 5, 10, 20, and 45 samples per group. Panels (e–h) show the same for DESeq2. Panel (i) shows directional overlap of significant genes between the tools at each sample size. Panel (j) shows DEG set stability under sample swapping (outlier simulation) using Jaccard similarity.
• Figure 2: Comparison of edgeR and DESeq2 across multiple biological contrasts. Panel (a) shows the log$_2$-scaled number of uniquely identified upregulated and downregulated genes by each tool across 13 contrasts spanning viral, bacterial, and fibrotic conditions. Panel (b) displays the Jaccard index for upregulated and downregulated gene sets, indicating overlap between tools. Panel (c) shows Pearson and Spearman correlation coefficients computed for Bonferroni-adjusted $p$-values among common significant genes. MPXV and EBOV comparisons are based on differential expression at specific days post-infection (DPI) relative to control samples.
• Figure 3: Classification performance of uniquely identified genes from edgeR and DESeq2. Each gene set was evaluated using a logistic regression classifier trained on PC1 and PC2 from log-transformed expression values. Panel (a) shows precision, (b) shows recall, and (c) shows F1 score for each dataset. Higher values indicate greater biological separability of control versus treated samples. Panel (d) shows the Dolan-More profile of both tools based on F1 scores, summarizing overall method robustness across all datasets.
• Figure 4: Cross-study generalizability of uniquely significant genes from edgeR and DESeq2. (a) Mean ROC curves across four independent SARS-CoV-2 datasets, with shaded regions representing $\pm 1$ standard deviation of the true positive rate (TPR) at each false positive rate (FPR). Metrics in the legend summarize mean AUC, accuracy, precision, and recall with standard deviation. (b) PCA-based classification using DESeq2-specific genes from training datasets applied to test set GSE152418, yielding moderate separation (AUC = 0.783). (c) Corresponding classification using edgeR-specific genes for the same test set, yielding perfect separation (AUC = 1.000).