Semiparametric Efficiency in Policy Learning with General Treatments

TL;DR

This paper develops a unified semiparametric efficiency framework for policy learning with general treatments, showing that deterministic policies are typically not pathwise differentiable and that randomized policies are essential for root-n estimation of welfare-based policy parameters. It derives the efficient influence function and the semiparametric efficiency bound for welfare under randomized policies, and reveals an Hirano–Imbens–Ridder-type phenomenon in policy learning: IPW with an estimated propensity achieves efficient regret, while IPW with the true propensity does not. The authors analyze three common policy estimators—IPW with known propensity, IPW with estimated propensity, and doubly robust estimators—showing that cross-fitting and orthogonality yield efficient regret under the estimated propensity and DR estimators. They validate the theory via empirically calibrated simulations on job training data and an empirical application to a commitment savings program, illustrating practical gains in regret efficiency and inference precision. The results provide concrete guidance for implementing efficient policy-learning procedures in settings with discrete, continuous, or mixed treatments.

Abstract

Recent literature on policy learning has primarily focused on regret bounds of the learned policy. We provide a new perspective by developing a unified semiparametric efficiency framework for policy learning, allowing for general treatments that are discrete, continuous, or mixed. We provide a characterization of the failure of pathwise differentiability for parameters arising from deterministic policies. We then establish efficiency bounds for pathwise differentiable parameters in randomized policies, both when the propensity score is known and when it must be estimated. Building on the convolution theorem, we introduce a notion of efficiency for the asymptotic distribution of welfare regret, showing that inefficient policy estimators not only inflate the variance of the asymptotic regret but also shift its mean upward. We derive the asymptotic theory of several common policy estimators, with a key contribution being a policy-learning analogue of the Hirano-Imbens-Ridder (HIR) phenomenon: the inverse propensity weighting estimator with an estimated propensity is efficient, whereas the same estimator using the true propensity is not. We illustrate the theoretical results with an empirically calibrated simulation study based on data from a job training program and an empirical application to a commitment savings program.

Figures (1)

• Figure 1: Density of normalized regret

Theorems & Definitions (22)

• Theorem 1
• Lemma 1
• Theorem 2
• Theorem 3
• Theorem 4
• Lemma 2
• Theorem 5
• Theorem 6
• Theorem 7
• proof : Proof of Theorem \ref{['thm:deterministic_irregular']}