Code Contribution and Credit in Science

TL;DR

This study constructs the Research Software Graph (rs-graph-v1) to quantify how software contributions relate to scientific credit. By linking 163k article-repository pairs (with a purified 138k one-to-one subset) and matching authors to developers via a transformer model (F1=0.944), the authors uncover that about $28.6 ext{ extpercent}$ of papers have non-author code contributors, and code-contributing authors show a modest $4.2 ext{ extpercent}$ citation advantage that vanishes when controlling for context. A striking pattern is that first authors are much more likely to contribute code (≈$69.8 ext{ extpercent}$) than middle or last authors, while frequent coding correlates with significantly lower h-indices, indicating a systematic misalignment between software labor and traditional scholarly credit. These results imply substantial policy implications for institutional reward structures and call for more equitable recognition of software-related contributions in science.

Abstract

Software development has become essential to scientific research, but its relationship to traditional metrics of scholarly credit remains poorly understood. We develop a dataset of approximately 140,000 paired research articles and code repositories, as well as a predictive model that matches research article authors with software repository developer accounts. We use this data to investigate how software development activities influence credit allocation in collaborative scientific settings. Our findings reveal significant patterns distinguishing software contributions from traditional authorship credit. We find that nearly 30% of articles include non-author code contributors- individuals who participated in software development but received no formal authorship recognition. While code-contributing authors show a modest $\sim$4.2% increase in article citations, this effect becomes non-significant when controlling for domain, article type, and open access status. First authors are significantly more likely to be code contributors than other author positions. Notably, we identify a negative relationship between coding frequency and scholarly impact metrics. Authors who contribute code more frequently exhibit progressively lower h-indices than non-coding colleagues, even when controlling for publication count, author position, domain, and article type. These results suggest a disconnect between software contributions and credit, highlighting important implications for institutional reward structures and science policy.

Figures (5)

• Figure 1: Data Gathering and Processing Workflow. We begin by gathering article-repository pairs from PLOS, JOSS, SoftwareX, and Papers with Code, retaining only pairs where the source code repository is hosted on GitHub. For each article-repository pair, we retrieve the DOI for the most recent version of the article via the Semantic Scholar API, then obtain the article's metadata, author information, and bibliometric data from OpenAlex. In parallel, we collect the source code repository's metadata and contributor information using the GitHub API. After retrieving both article and repository information, we apply our trained author-developer matching model across all possible combinations of article authors and repository contributors for each article-repository pair. Finally, we store all retrieved information and the matched author-developer pairs in a SQLite database. Article-repository pair counts are approximate because there are no snapshots of these databases at a single point in time. The estimates are based on counts from each data source taken in October 2025.
• Figure 2: Mean of Non-Code-Contributing Authors (NCC-A), Code-Contributing Authors (CC-A), and Code-Contributing Non-Authors (CC-NA) Research Team Members by Domain, Article Type, and Open Access Status. Only includes research teams from article-repository pairs with repositories that have programming language files, a most recent commit no later than 90 days after publication, and excludes research teams in the top 3% of total author sizes.
• Figure 3: Average number of contributors per article, by contribution type, along with A) the year the article was published and B) the total number of authors included in the article. Only includes research teams from article-repository pairs with repositories that have programming language files, a most recent commit no later than 90 days after publication, and excludes research teams in the top 3% of total author sizes for publication years with 50 or more articles. Shaded areas show the 95% confidence interval for the mean.
• Figure 4: Distribution of author h-index by coding frequency across three key publication factors. Results are grouped by each author's most frequent: (1) position in publication bylines (first, middle, or last), (2) publication type (preprint, research article, or software article), and (3) research domain (Social Sciences, Physical Sciences, Health Sciences, or Life Sciences). Within each subplot, h-indices are divided by the author's coding frequency: 'none' (no coding in any of their publications), 'any' (coding in at least one but fewer than half of their publications), 'majority' (coding in at least half but not all of their publications), and 'always' (coding in each of their publications). Authors are only included if they have three or more publications within our dataset and are associated with no more than three developer accounts, with each association having a predicted model confidence of at least 97%.
• Figure 5: Distribution of author-developer-account prediction confidence scores. The plot on the left shows the distribution of all prediction confidence scores. The plot on the right shows the distribution of prediction confidence scores for author-developer-account pairs with a confidence score greater than or equal to 0.97.