Generative AI and Firm Productivity: Field Experiments in Online Retail

TL;DR

This study provides causal evidence on GenAI's firm-level productivity effects in online retail using seven large-scale field experiments. By holding inputs constant, the authors interpret output gains as total factor productivity improvements driven by improved consumer experience and reduced frictions, with conversion rates rising while average cart values remain stable. They find substantial but heterogeneous gains across workflows, with the largest effects in customer-service and search-related tasks, and notable heterogeneity across sellers, buyers, and products, particularly favoring smaller sellers and less experienced consumers. Aggregating across four positive workflows, the research estimates an annual incremental value of about $4.6–$5 per consumer, underscoring GenAI's potential to generate meaningful revenue-based productivity at scale. The work highlights both the promise and the caveats of GenAI adoption, including the need for domain-specific fine-tuning, longer-horizon analyses, and consideration of general-equilibrium effects as adoption broadens.

Abstract

We quantify the impact of Generative Artificial Intelligence (GenAI) on firm productivity through a series of large-scale randomized field experiments involving millions of users and products at a leading cross-border online retail platform. Over six months in 2023-2024, GenAI-based enhancements were integrated into seven consumer-facing business workflows. We find that GenAI adoption significantly increases sales, with treatment effects ranging from $0\%$ to $16.3\%$, depending on GenAI's marginal contribution relative to existing firm practices. Because inputs and prices were held constant across experimental arms, these gains map directly into total factor productivity improvements. Across the four GenAI applications with positive effects, the implied annual incremental value is approximately $\$ 5$ per consumer-an economically meaningful impact given the retailer's scale and the early stage of GenAI adoption. The primary mechanism operates through higher conversion rates, consistent with GenAI reducing frictions in the marketplace and improving consumer experience. We also document substantial heterogeneity: smaller and newer sellers, as well as less experienced consumers, exhibit disproportionately larger gains. Our findings provide novel, large-scale causal evidence on the productivity effects of GenAI in online retail, highlighting both its immediate value and broader potential.

Figures (8)

• Figure 1: P-Values for Covariate Balance Checks Across Experiments
• Figure A1: Illustration of Pre-sale Service Chatbot
• Figure A2: Illustration of Search Query Refinement
• Figure A3: Illustration of Product Description
• Figure A4: Illustration of Marketing Push Message