Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Early Impacts of M365 Copilot

Eleanor Wiske Dillon, Sonia Jaffe, Sida Peng, Alexia Cambon

TL;DR

This study provides early evidence on how access to a generative AI assistant embedded in a widely used office suite affects knowledge-work patterns. Using a large randomized field experiment with a difference-in-differences framework and instrumental-variable analyses, it documents meaningful time savings in email reading and document production, while meeting-time effects are heterogeneous and less clear. Adoption levels and firm-specific contexts strongly influence outcomes, suggesting that broader rollout and training will be crucial to realizing substantial gains. Overall, Copilot shows potential to reduce time spent on core tasks, indicating a path toward more efficient knowledge work as adoption scales.

Abstract

Advances in generative AI have rapidly expanded the potential of computers to perform or assist in a wide array of tasks traditionally performed by humans. We analyze a large, real-world randomized experiment of over 6,000 workers at 56 firms to present some of the earliest evidence on how these technologies are changing the way knowledge workers do their jobs. We find substantial time savings on common core tasks across a wide range of industries and occupations: workers who make use of this technology spent half an hour less reading email each week and completed documents 12% faster. Despite the newness of the technology, nearly 40% of workers who were given access to the tool used it regularly in their work throughout the 6-month study.

Early Impacts of M365 Copilot

TL;DR

This study provides early evidence on how access to a generative AI assistant embedded in a widely used office suite affects knowledge-work patterns. Using a large randomized field experiment with a difference-in-differences framework and instrumental-variable analyses, it documents meaningful time savings in email reading and document production, while meeting-time effects are heterogeneous and less clear. Adoption levels and firm-specific contexts strongly influence outcomes, suggesting that broader rollout and training will be crucial to realizing substantial gains. Overall, Copilot shows potential to reduce time spent on core tasks, indicating a path toward more efficient knowledge work as adoption scales.

Abstract

Advances in generative AI have rapidly expanded the potential of computers to perform or assist in a wide array of tasks traditionally performed by humans. We analyze a large, real-world randomized experiment of over 6,000 workers at 56 firms to present some of the earliest evidence on how these technologies are changing the way knowledge workers do their jobs. We find substantial time savings on common core tasks across a wide range of industries and occupations: workers who make use of this technology spent half an hour less reading email each week and completed documents 12% faster. Despite the newness of the technology, nearly 40% of workers who were given access to the tool used it regularly in their work throughout the 6-month study.

Paper Structure

This paper contains 9 sections, 1 equation, 4 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Experiment Design and Implementation
  3. Methods
  4. Results
  5. Email
  6. Documents
  7. Meetings
  8. Conclusion
  9. Results by Industry

Figures (4)

  • Figure 1: Firms Participating In The Experiment Over Time
  • Figure 2: Active Copilot Usage – All Firms and Median
  • Figure 3: Copilot Usage by App - Across Firms
  • Figure 4: Effect on Total Time in Teams Meetings by Firm Avg. Pre-period Weekly Meetings