The Value of Personalized Recommendations: Evidence from Netflix

TL;DR

This paper addresses how to disentangle intrinsic demand for goods from the effects of personalized recommendations on a large streaming platform. It develops a low-rank, discrete-choice model with an additive recommendation bonus and a transformer-style state-dependence mechanism to capture evolving user tastes, estimated on ~2 million U.S. Netflix users with endogeneity addressed via exogenous variation and model-free diversion ratios for validation. The authors quantify the value of the current RecSys relative to benchmarks (Random, Popular, Matrix Factorization) and decompose effects into selection, exposure, and targeting, finding that targeting drives most incremental engagement, particularly for mid-popularity goods. The approach provides a framework for measuring incremental demand for goods and offers actionable insights for catalog optimization and recommender-system design, highlighting the importance of personalization beyond mere exposure.

Abstract

Personalized recommendation systems shape much of user choice online, yet their targeted nature makes separating out the value of recommendation and the underlying goods challenging. We build a discrete choice model that embeds recommendation-induced utility, low-rank heterogeneity, and flexible state dependence and apply the model to viewership data at Netflix. We exploit idiosyncratic variation introduced by the recommendation algorithm to identify and separately value these components as well as to recover model-free diversion ratios that we can use to validate our structural model. We use the model to evaluate counterfactuals that quantify the incremental engagement generated by personalized recommendations. First, we show that replacing the current recommender system with a matrix factorization or popularity-based algorithm would lead to 4% and 12% reduction in engagement, respectively, and decreased consumption diversity. Second, most of the consumption increase from recommendations comes from effective targeting, not mechanical exposure, with the largest gains for mid-popularity goods (as opposed to broadly appealing or very niche goods).

Figures (11)

• Figure 1: Netflix Homepage
• Figure 2: Preference Weight ($A_{it}$) Sequence Model
• Figure 3: Most and Least Similar Goods for Selected Goods
• Figure 4: UMAP-projected Good Embeddings
• Figure 5: Observed Good Shares vs In-Sample Model Estimates