The Heterogeneous Productivity Effects of Generative AI
David Kreitmeir, Paul A. Raschky
TL;DR
The study tackles how generative AI tools influence knowledge-worker productivity by exploiting Italy's sudden ChatGPT ban as a natural experiment. It analyzes high-frequency GitHub activity from 36,358 developers across treatment (Italy) and control countries, using a difference-in-differences framework and event-study designs to trace dynamics around the ban, with robustness checks including repository-level analyses and placebo tests. The main finding is heterogeneous: less experienced developers show a short-term boost in both output quantity and quality after the ban, while more experienced developers show no systematic productivity gains, and package contributors exhibit some adverse effects on routine tasks. The work highlights that AI-enabled productivity gains are not uniform across experience levels, underscores potential policy costs of abrupt digital bans, and argues for targeted AI-use safeguards (e.g., guard rails in tools like Copilot) to balance productivity gains against error-prone outputs.
Abstract
We analyse the individual productivity effects of Italy's ban on ChatGPT, a generative pretrained transformer chatbot. We compile data on the daily coding output quantity and quality of over 36,000 GitHub users in Italy and other European countries and combine these data with the sudden announcement of the ban in a difference-in-differences framework. Among the affected users in Italy, we find a short-term increase in output quantity and quality for less experienced users and a decrease in productivity on more routine tasks for experienced users.