Logarithmic Neyman Regret for Adaptive Estimation of the Average Treatment Effect

TL;DR

The Clipped Second Moment Tracking algorithm is proposed, a variant of an existing algorithm with strong asymptotic optimality guarantees, and it is shown that ClipSMT achieves exponential improvements in Neyman regret on two fronts: improving the dependence on $T$ from $O(\sqrt{T})$ to $O(\log T)$, as well as reducing the exponential dependence on problem parameters to a polynomial dependence.

Abstract

Estimation of the Average Treatment Effect (ATE) is a core problem in causal inference with strong connections to Off-Policy Evaluation in Reinforcement Learning. This paper considers the problem of adaptively selecting the treatment allocation probability in order to improve estimation of the ATE. The majority of prior work on adaptive ATE estimation focus on asymptotic guarantees, and in turn overlooks important practical considerations such as the difficulty of learning the optimal treatment allocation as well as hyper-parameter selection. Existing non-asymptotic methods are limited by poor empirical performance and exponential scaling of the Neyman regret with respect to problem parameters. In order to address these gaps, we propose and analyze the Clipped Second Moment Tracking (ClipSMT) algorithm, a variant of an existing algorithm with strong asymptotic optimality guarantees, and provide finite sample bounds on its Neyman regret. Our analysis shows that ClipSMT achieves exponential improvements in Neyman regret on two fronts: improving the dependence on $T$ from $O(\sqrt{T})$ to $O(\log T)$, as well as reducing the exponential dependence on problem parameters to a polynomial dependence. Finally, we conclude with simulations which show the marked improvement of ClipSMT over existing approaches.

Figures (2)

• Figure 1: Comparison of the performance of ClipSMT, ClipOGD, Explore-then-Commit (EtC), Neyman allocation, and a balanced allocation with the treatment and control arms following Bernoulli distributions. Individual subplots plot the variance of each design against the number of samples for a fixed problem instance. Each column keeps the treatment mean fixed, and each row keeps the Neyman allocation fixed. Moving to the right increases the treatment mean and moving down increases the Neyman allocation. Overall the performance of ClipSMT is always competitive with the performance of the infeasible Neyman allocation and outperforms the other adaptive designs. Furthermore, as the Neyman allocation increases, we see that ClipSMT adapts to the increased difficulty while EtC and the balanced design do not.
• Figure 2: Plot of the ratio between the clipping time predicted by Lemma \ref{['lem:clipping-phase-length-bound']} and the empirically computed clipping time against different values for $\alpha$. Each line represents the ratios for a different problem instance. We see that for most problem instances the ratio remains relatively constant for moderate values of $\alpha$.

Theorems & Definitions (32)

• Theorem 4.1
• proof
• proof
• proof
• Lemma A.1
• proof
• Lemma B.1
• proof
• Lemma B.2
• proof