Generative AI Uses and Risks for Knowledge Workers in a Science Organization

TL;DR

Generative AI offers the potential to accelerate scientific knowledge work, yet its real-world workplace application and risks remain unclear. This study at Argonne National Lab uses a mixed-methods design (survey N=66, interviews N=22, Argo usage data) to map how Science and Operations staff use and imagine AI copilots and workflow agents, revealing an upward but still modest adoption and two dominant modalities: copilot for writing and data querying, and workflow agents for automated tasks and pipelines. Key contributions include a novel organizational case study of private-instance LLM deployment, a taxonomy of current and envisioned AI use cases across roles, and design/policy recommendations to mitigate reliability, privacy, and publishing concerns while guiding hiring and skills development. The findings inform practical guidelines for implementing AI in science organizations and offer a framework for analyzing AI risks in knowledge-work settings more broadly, with implications for governance, transparency, and future research in HCI and CSCW.

Abstract

Generative AI could enhance scientific discovery by supporting knowledge workers in science organizations. However, the real-world applications and perceived concerns of generative AI use in these organizations are uncertain. In this paper, we report on a collaborative study with a US national laboratory with employees spanning Science and Operations about their use of generative AI tools. We surveyed 66 employees, interviewed a subset (N=22), and measured early adoption of an internal generative AI interface called Argo lab-wide. We have four findings: (1) Argo usage data shows small but increasing use by Science and Operations employees; Common current and envisioned use cases for generative AI in this context conceptually fall into either a (2) copilot or (3) workflow agent modality; and (4) Concerns include sensitive data security, academic publishing, and job impacts. Based on our findings, we make recommendations for generative AI use in science and other organizations.

Figures (5)

• Figure 1: Argo usage metrics by Operations and Science unique users for each month since initial deployment. Monthly usage is less than 10% of all lab employees. Note: this plot does not capture employee use of external LLMs and is based on auto-collected telemetry data.
• Figure 2: Responses to the survey question: How familiar are you with large language models (LLMs) such as ChatGPT, Argo, etc.?
• Figure 3: Responses to the survey question: To what extent would you agree/disagree with the following statement: LLMs have become an essential part of my workflow.
• Figure 4: Responses to the survey questions: How often do you use LLMs as part of your work? and How often do you use LLMs for personal use?
• Figure 5: Responses to the survey question: How often do you use LLMs for the following Argonne National Lab-related work tasks?