Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reproducibility, Replicability, and Transparency in Research: What 430 Professors Think in Universities across the USA and India

Tatiana Chakravorti, Sai Dileep Koneru, Sarah Rajtmajer

TL;DR

This study addresses how researchers in the USA and India perceive reproducibility, replicability, and open science. Using a survey of 430 professors across social sciences and engineering, it maps awareness, experiences with replication, attitudes toward open science, and institutional barriers. The findings reveal persistent incentives and resource constraints that hinder reliable, transparent research, with notable cross-country and disciplinary differences. The authors argue for culturally centered solutions, including revised incentive structures, targeted education, and improved peer-review practices to advance scientific integrity. Overall, the work highlights practical pathways to strengthen reproducibility by aligning norms, resources, and recognition with open-science principles.

Abstract

In the past decade, open science and science of science communities have initiated innovative efforts to address concerns about the reproducibility and replicability of published scientific research. In some respects, these efforts have been successful, yet there are still many pockets of researchers with little to no familiarity with these concerns, subsequent responses, or best practices for engaging in reproducible, replicable, and reliable scholarship. In this work, we survey 430 professors from Universities across the USA and India to understand perspectives on scientific processes and identify key points for intervention. Our findings reveal both national and disciplinary gaps in attention to reproducibility and replicability, aggravated by incentive misalignment and resource constraints. We suggest that solutions addressing scientific integrity should be culturally-centered, where definitions of culture should include both regional and domain-specific elements.

Reproducibility, Replicability, and Transparency in Research: What 430 Professors Think in Universities across the USA and India

TL;DR

This study addresses how researchers in the USA and India perceive reproducibility, replicability, and open science. Using a survey of 430 professors across social sciences and engineering, it maps awareness, experiences with replication, attitudes toward open science, and institutional barriers. The findings reveal persistent incentives and resource constraints that hinder reliable, transparent research, with notable cross-country and disciplinary differences. The authors argue for culturally centered solutions, including revised incentive structures, targeted education, and improved peer-review practices to advance scientific integrity. Overall, the work highlights practical pathways to strengthen reproducibility by aligning norms, resources, and recognition with open-science principles.

Abstract

In the past decade, open science and science of science communities have initiated innovative efforts to address concerns about the reproducibility and replicability of published scientific research. In some respects, these efforts have been successful, yet there are still many pockets of researchers with little to no familiarity with these concerns, subsequent responses, or best practices for engaging in reproducible, replicable, and reliable scholarship. In this work, we survey 430 professors from Universities across the USA and India to understand perspectives on scientific processes and identify key points for intervention. Our findings reveal both national and disciplinary gaps in attention to reproducibility and replicability, aggravated by incentive misalignment and resource constraints. We suggest that solutions addressing scientific integrity should be culturally-centered, where definitions of culture should include both regional and domain-specific elements.
Paper Structure (23 sections, 5 figures)

This paper contains 23 sections, 5 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Reproducibility and the Replication Crisis
  4. Open Science
  5. Methods
  6. Survey recruitment
  7. Data collection and analysis
  8. Findings
  9. Awareness and concern about reproducibility and replicability (RQ1)
  10. Factors contributing to lack of reproducibility
  11. Experiences reproducing and replicating others' findings (RQ1)
  12. Insufficient detail provided in the paper
  13. Conflicting results
  14. Affirmative results
  15. Hopeful progress
  16. ...and 8 more sections

Figures (5)

  • Figure 1: Survey responses by country and domain (Eng=engineering; SS=social science)
  • Figure 2: Factors contributing to lack of reproducibility
  • Figure 3: Opinions about the open science movement
  • Figure 4: Participants were asked whether changes to publication, funding, or promotional models (inclusive) were needed to advance reproducibility.
  • Figure 5: Signals of credibility