Model-Agnostic Bounds for Augmented Inverse Probability Weighted Estimators' Wald-Confidence Interval Coverage in Randomized Controlled Trials

TL;DR

This work analyzes Wald-confidence-interval coverage for augmented inverse probability weighted estimators in randomized trials, focusing on nonparametric nuisance estimators and a known propensity score. It derives non-asymptotic, model-agnostic Berry-Esseen-type bounds for CI coverage, with explicit dependence on cross-fitting, nuisance-estimator convergence, and function-class complexity. The study reveals that cross-fitting can improve CI coverage through reduced empirical-process error and potential overestimation of variance, while non-cross-fitting may lead to slower or biased coverage, especially under rich nuisance-function classes. Simulations corroborate that cross-fit AIPW methods, particularly CVSL, offer favorable variance behavior and coverage near the nominal level in moderate samples, providing practical guidance for causal inference in RCT settings with flexible nuisance estimation.

Abstract

Nonparametric estimators, such as the augmented inverse probability weighted (AIPW) estimator, have become increasingly popular in causal inference. Numerous nonparametric estimators have been proposed, but they are all asymptotically normal with the same asymptotic variance under similar conditions, leaving little guidance for practitioners to choose an estimator. In this paper, I focus on another important perspective of their asymptotic behaviors beyond asymptotic normality, the convergence of the Wald-confidence interval (CI) coverage to the nominal coverage. Such results have been established for simpler estimators (e.g., the Berry-Esseen Theorem), but are lacking for nonparametric estimators. I consider a simple but practical setting where the AIPW estimator based on a black-box nuisance estimator, with or without cross-fitting, is used to estimate the average treatment effect in randomized controlled trials. I derive non-asymptotic Berry-Esseen-type bounds on the difference between Wald-CI coverage and the nominal coverage. I also analyze the bias of variance estimators, showing that the cross-fit variance estimator might overestimate while the non-cross-fit variance estimator might underestimate, which might explain why cross-fitting has been empirically observed to improve Wald-CI coverage.

Figures (5)

• Figure 1: Sampling distribution of ATE estimators. The horizontal blue line is the true ATE.
• Figure 2: Sampling distribution of estimated influence function-based asymptotic variance for AIPW estimators (similar to $\hat{\sigma}_a^2$ and $\tilde{\sigma}_a^2$). The violin-like shapes are vertical densities of the estimated variance. The black dots are the Monte Carlo variance of ATE estimators, scaled by $n$ (similar to $n {\mathrm{var}}(\hat{\psi}_a)$ and $n {\mathrm{var}}(\tilde{\psi}_a)$). The blue triangles are the Monte Carlo average of the estimated variance (similar to $\sigma_{\dagger,a}^2$). The red crosses are the oracle variance (similar to $\sigma_{\#,a}^2$). The horizontal gray line is the efficient asymptotic variance (similar to $\sigma_{*,a}^2$).
• Figure 3: QQ-plot of AIPW estimators. The y-axis is the Monte Carlo sample quantile. The x-axis is the theoretical quantile of a normal distribution with mean being true ATE $\psi_{*,1}-\psi_{*,0}$ and variance being the oracle variance (similar to $\sigma_{\#,a}^2$). The blue line is the diagonal line $y=x$.
• Figure 4: CI coverage with 95% Wilson confidence intervals of coverage. The horizontal blue line is 95%, the nominal coverage. Subfigure (B) is a zoomed version of Subfigure (A) with SL removed.
• Figure S1: The factor concerning $K$ in the cross-fit Wald-CI coverage's convergence rate to the nominal coverage. The number of folds $K$ ranges over even numbers between 2 and 20. The rate $r$ ranges from 0.4 (slow) to 1 (parametric). Subfigures (A) and (B) are for the cases with and without approximate sub-Weibull conditions \ref{['cond: CV tail']} and \ref{['cond: more CV tail']}, respectively. The points are dodged horizontally.

Theorems & Definitions (47)

• Theorem 1: Asymptotic Berry-Esseen-type bound for cross-fit AIPW Wald-CI
• Theorem 2: Convergence rate of $|\sigma_{\dagger,a} - \sigma_{\#,a}|$ with cross-fitting
• Theorem 3: Asymptotic Berry-Esseen-type bound for non-cross-fit AIPW Wald-CI
• Theorem 4: Convergence rate of $|\sigma_{\dagger,a} - \sigma_{\#,a}|$ without cross-fitting
• Corollary 1: Convergence rates of non-cross-fit AIPW Wald-CI coverage for VC classes and VC-hulls
• Proposition 1: Asymptotic order of bias of variance estimators
• Proposition 2
• Theorem S1: Non-asymptotic Berry-Esseen-type bound for cross-fit AIPW Wald-CI
• Theorem S2: Bound on $|\sigma_{\dagger,a} - \sigma_{\#,a}|$ with cross-fitting
• Theorem S3: Non-asymptotic Berry-Esseen-type bound for non-cross-fit AIPW Wald-CI