A survey of generative AI adoption and perceived productivity among scientists who program

Abstract

Programming is essential to modern scientific research, yet most scientists report inadequate training for the software development their work demands. Generative AI tools capable of code generation may support scientific programmers, but user studies indicate risks of over-reliance, particularly among inexperienced users. We surveyed 868 scientists who program, examining adoption patterns, tool preferences, and factors associated with perceived productivity. Adoption is highest among students and less experienced programmers, with variation across fields. Scientific programmers overwhelmingly prefer general-purpose conversational interfaces like ChatGPT over developer-specific tools. Both inexperience and limited use of development practices (like testing, code review, and version control) are associated with greater perceived productivity -- but these factors interact, suggesting formal practices may partially compensate for inexperience. The strongest predictor of perceived productivity is the number of lines of generated code typically accepted at once. These findings suggest scientific programmers using generative AI may gauge productivity by code generation rather than validation.

Figures (14)

• Figure 1: Distribution of 760 responses to survey question, How often do you use a genAI tool in your research-related programming?
• Figure 2: Generative AI tool usage frequency for research-related programming by research area categories. Labels on bars indicate counts, and the $x$-axis indicates proportions. There were 760 responses to this question, and all respondents also provided a research area.
• Figure 3: Generative AI tool usage frequency for research-related programming by role. Labels on bars indicate counts.
• Figure 4: Familiarity with and usage frequency of software development practices among survey respondents. Labels on bars indicate counts, and the $x$-axis indicates proportions. Note that respondents may skip survey items, so the number of total responses per practice can differ.
• Figure 5: Pairwise correlation matrix for perceived productivity score and variables related to development practices and programming experience. The matrix is ordered via hierarchical clustering.