An analysis of the effects of sharing research data, code, and preprints on citations

TL;DR

This study examines how Open Science practices—data sharing, code sharing, and preprint posting—relate to citation impact, using a large, reproducible analysis of about 121,999 publications drawn from the OSI dataset (PLOS 2018–2023 plus a PMC Open Access Subset comparator). It employs log-transformed, regression-based models with extensive controls, including division fixed effects, to isolate the relative effects of each practice. The main findings show that preprints are associated with a significant citation advantage of about $20.2$–$20.4\%$, and data sharing via online repositories with about $4\%$, while code sharing shows no statistically significant effect. The results underscore disciplinary variation and emphasize the need for causal and broader-impact research beyond citations; they also advocate transparent data and code availability to support replication and extension.

Abstract

Calls to make scientific research more open have gained traction with a range of societal stakeholders. Open Science practices include but are not limited to the early sharing of results via preprints and openly sharing outputs such as data and code to make research more reproducible and extensible. Existing evidence shows that adopting Open Science practices has effects in several domains. In this study, we investigate whether adopting one or more Open Science practices leads to significantly higher citations for an associated publication, which is one form of academic impact. We use a novel dataset known as Open Science Indicators, produced by PLOS and DataSeer, which includes all PLOS publications from 2018 to 2023 as well as a comparison group sampled from the PMC Open Access Subset. In total, we analyze circa 122'000 publications. We calculate publication and author-level citation indicators and use a broad set of control variables to isolate the effect of Open Science Indicators on received citations. We show that Open Science practices are adopted to different degrees across scientific disciplines. We find that the early release of a publication as a preprint correlates with a significant positive citation advantage of about 20.2% on average. We also find that sharing data in an online repository correlates with a smaller yet still positive citation advantage of 4.3% on average. However, we do not find a significant citation advantage for sharing code. Further research is needed on additional or alternative measures of impact beyond citations. Our results are likely to be of interest to researchers, as well as publishers, research funders, and policymakers.

Figures (4)

• Figure 1: Correlation plot among most variables. We see that no two variables are too highly correlated, except as expected for two alternatives for dependent variables (n_cit_2 and n_cit_tot).
• Figure 2: Adoption of OSI over time. Each OSI remains adopted by a fraction of publications, but adoption grows over time.
• Figure 3: Adoption of OSI by Division, as described in Table \ref{['tab:ANZSRC']}. Each OSI remains adopted by a fraction of publications, but there is a wide variation across Divisions.
• Figure 4: An illustration of the assumed causal dependency graph among dependent and independent variables. We distinguish among the dependent variable and its variations (red), independent control variables (blue), and OSI control variables (green). We are interested in the total effect of OSI variables on the dependent variable (n_cit_tot), shown by the thick black line, and in controlling for the effect of other independent variables, shown by the dotted black lines.