Analyzing GitHub Issues and Pull Requests in nf-core Pipelines: Insights into nf-core Pipeline Repositories
Khairul Alam, Banani Roy
TL;DR
The paper analyzes 25,173 GitHub issues and PRs from 125 nf-core pipelines to uncover recurring challenges and collaboration patterns using BERTopic topic modeling. It identifies 13 topics spanning the entire pipeline lifecycle and shows strong resolution dynamics, with 89.38% of items closed and half closed within three days; active labeling and inclusion of code blocks significantly improve closure likelihood. Tool development, CI configuration, and genome data integration emerge as the most difficult areas, signaling needs for automation, better diagnostics, and clearer contributor guidance. The study provides actionable implications for improving usability, sustainability, and reproducibility in nf-core pipelines and offers a replication package to support reproducibility and broader comparisons across domains.
Abstract
Scientific Workflow Management Systems (SWfMSs) such as Nextflow have become essential software frameworks for conducting reproducible, scalable, and portable computational analyses in data-intensive fields like genomics, transcriptomics, and proteomics. Building on Nextflow, the nf-core community curates standardized, peer-reviewed pipelines that follow strict testing, documentation, and governance guidelines. Despite its broad adoption, little is known about the challenges users face during the development and maintenance of these pipelines. This paper presents an empirical study of 25,173 issues and pull requests from these pipelines to uncover recurring challenges, management practices, and perceived difficulties. Using BERTopic modeling, we identify 13 key challenges, including pipeline development and integration, bug fixing, integrating genomic data, managing CI configurations, and handling version updates. We then examine issue resolution dynamics, showing that 89.38\% of issues and pull requests are eventually closed, with half resolved within three days. Statistical analysis reveals that the presence of labels (large effect, $δ$ = 0.94) and code snippets (medium effect, $δ$ = 0.50) significantly improve resolution likelihood. Further analysis reveals that tool development and repository maintenance poses the most significant challenges, followed by testing pipelines and CI configurations, and debugging containerized pipelines. Overall, this study provides actionable insights into the collaborative development and maintenance of nf-core pipelines, highlighting opportunities to enhance their usability, sustainability, and reproducibility.