Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Current and Future Use of Large Language Models for Knowledge Work

Michelle Brachman, Amina El-Ashry, Casey Dugan, Werner Geyer

TL;DR

Knowledge workers increasingly rely on Large Language Models (LLMs) to augment and automate tasks. The authors conduct two waves of surveys within a large tech company to categorize current LLM usage into creation, information, advice, and automation, and to map how workers want to use LLMs in the future, including greater automation and deeper integration with personal and organizational data. Results show LLM usage often forms part of larger, collaborative workflows and requires oversight, transparency, and data/tool integration. These findings inform design directions for enterprise LLM tools, emphasize workflow-aware integration, and point to future research on collaborative LLM use and trust in the workplace.

Abstract

Large Language Models (LLMs) have introduced a paradigm shift in interaction with AI technology, enabling knowledge workers to complete tasks by specifying their desired outcome in natural language. LLMs have the potential to increase productivity and reduce tedious tasks in an unprecedented way. A systematic study of LLM adoption for work can provide insight into how LLMs can best support these workers. To explore knowledge workers' current and desired usage of LLMs, we ran a survey (n=216). Workers described tasks they already used LLMs for, like generating code or improving text, but imagined a future with LLMs integrated into their workflows and data. We ran a second survey (n=107) a year later that validated our initial findings and provides insight into up-to-date LLM use by knowledge workers. We discuss implications for adoption and design of generative AI technologies for knowledge work.

Current and Future Use of Large Language Models for Knowledge Work

TL;DR

Knowledge workers increasingly rely on Large Language Models (LLMs) to augment and automate tasks. The authors conduct two waves of surveys within a large tech company to categorize current LLM usage into creation, information, advice, and automation, and to map how workers want to use LLMs in the future, including greater automation and deeper integration with personal and organizational data. Results show LLM usage often forms part of larger, collaborative workflows and requires oversight, transparency, and data/tool integration. These findings inform design directions for enterprise LLM tools, emphasize workflow-aware integration, and point to future research on collaborative LLM use and trust in the workplace.

Abstract

Large Language Models (LLMs) have introduced a paradigm shift in interaction with AI technology, enabling knowledge workers to complete tasks by specifying their desired outcome in natural language. LLMs have the potential to increase productivity and reduce tedious tasks in an unprecedented way. A systematic study of LLM adoption for work can provide insight into how LLMs can best support these workers. To explore knowledge workers' current and desired usage of LLMs, we ran a survey (n=216). Workers described tasks they already used LLMs for, like generating code or improving text, but imagined a future with LLMs integrated into their workflows and data. We ran a second survey (n=107) a year later that validated our initial findings and provides insight into up-to-date LLM use by knowledge workers. We discuss implications for adoption and design of generative AI technologies for knowledge work.

Paper Structure

This paper contains 38 sections, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Adoption of LLMs
  4. Populations of adopters
  5. Types of usages
  6. Conversational Agents, Chatbots, and Intelligent Agents at Work
  7. Enterprise Knowledge and Information Workers
  8. Methods
  9. Survey
  10. Design Process
  11. Survey Questions
  12. Participants
  13. Survey1
  14. Survey2
  15. Data and Analysis
  16. ...and 23 more sections

Figures (3)

  • Figure 1: High-level survey flow, with topics discussed based on question of how they used LLMs.
  • Figure 2: Frequency of Survey 1 task types, based on inductive analysis of participants' open-ended descriptions of the tasks they currently use LLMs for and tasks they would like to use LLMs for in the future. Note: participants were not asked to describe all tasks, so these frequencies of description cannot be directly compared to Survey2.
  • Figure 3: Frequency of Survey2 task types, based on multiple choice (select all) questions asking which types of tasks participants currently use LLMs for and which tasks they would like to use LLMs for in the future.