Personalized Policy Learning through Discrete Experimentation: Theory and Empirical Evidence
Zhiqi Zhang, Zhiyu Zeng, Ruohan Zhan, Dennis Zhang
TL;DR
The paper tackles the challenge of learning personalized policies over a continuous treatment space when only discrete A/B arms are observed. It introduces DLPT, a three-stage framework that first estimates a high-dimensional nuisance function via structured neural networks, then uses Neyman orthogonal scores for unbiased policy-value estimation, and finally learns the optimal personalized policy within flexible policy classes. The authors establish theoretical guarantees, including asymptotic unbiasedness, semi-parametric efficiency, and a minimax $\sqrt{n}$ regret rate for the learned policy, and validate the method with a large field experiment on Platform O, showing substantial gains over benchmarks in both policy evaluation and policy learning. Complementary semi-synthetic experiments corroborate robustness across continuous policy spaces and provide practical guidance on design choices, such as treatment coverage and polynomial degree. The work offers a scalable, post-experiment tool for extrapolating continuous, personalized decisions from discrete experiments with real-world significance for platform operations.
Abstract
Randomized Controlled Trials (RCTs), or A/B testing, have become the gold standard for optimizing various operational policies on online platforms. However, RCTs on these platforms typically cover a limited number of discrete treatment levels, while the platforms increasingly face complex operational challenges involving optimizing continuous variables, such as pricing and incentive programs. The current industry practice involves discretizing these continuous decision variables into several treatment levels and selecting the optimal discrete treatment level. This approach, however, often leads to suboptimal decisions as it cannot accurately extrapolate performance for untested treatment levels and fails to account for heterogeneity in treatment effects across user characteristics. This study addresses these limitations by developing a theoretically solid and empirically verified framework to learn personalized continuous policies based on high-dimensional user characteristics, using observations from an RCT with only a discrete set of treatment levels. Specifically, we introduce a deep learning for policy targeting (DLPT) framework that includes both personalized policy value estimation and personalized policy learning. We prove that our policy value estimators are asymptotically unbiased and consistent, and the learned policy achieves a root-n-regret bound. We empirically validate our methods in collaboration with a leading social media platform to optimize incentive levels for content creation. Results demonstrate that our DLPT framework significantly outperforms existing benchmarks, achieving substantial improvements in both evaluating the value of policies for each user group and identifying the optimal personalized policy.