Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Automatic Debiased Machine Learning for Smooth Functionals of Nonparametric M-Estimands

Lars van der Laan, Aurelien Bibaut, Nathan Kallus, Alex Luedtke

TL;DR

This work introduces automatic debiased machine learning (autoDML) for inference on smooth functionals of nonparametric M-estimands, unifying and automating debiasing through Neyman orthogonality and a Hessian-induced Riesz representer. By reducing inference to two learning tasks—estimating the M-estimand and the Hessian representer—the framework yields estimators (one-step, TMLE, sieve) that are regular, asymptotically linear, and, under suitable conditions, semiparametrically efficient for the target functional. A functional von Mises expansion characterizes the efficient influence function as $\chi_0(z) = -\dot{\ell}_{\eta_0}(\theta_0)(\alpha_0)(z) + m(z, \theta_0) - \Psi(P_0)$, with the Hessian Riesz representer $\alpha_0$ identified via a risk-minimization problem; this underpins automatic bias corrections and confidence interval construction. The approach extends existing autoDML beyond regression functionals to functionals of vector-valued M-estimands and functionals that depend on nuisance components, achieving robustness to mild misspecification and enabling adaptive model selection through autoSieve. Empirical illustration on a beta-geometric survival model shows favorable finite-sample performance, highlighting the practical impact for complex semiparametric inference in causal settings and survival analysis.

Abstract

We develop a unified framework for automatic debiased machine learning (autoDML) to simplify inference for a broad class of statistical parameters. It applies to any smooth functional of a nonparametric \emph{M-estimand}, defined as the minimizer of a population risk over an infinite-dimensional linear space. Examples of M-estimands include counterfactual regression, quantile, and survival functions, as well as conditional average treatment effects. Rather than requiring manual derivation of influence functions, the framework automates the construction of debiased estimators using three components: the gradient and Hessian of the loss function and a linear approximation of the target functional. Estimation reduces to solving two risk minimization problems -- one for the M-estimand and one for a Riesz representer. The framework accommodates Neyman-orthogonal loss functions depending on nuisance parameters and extends to vector-valued M-estimands through joint risk minimization. For functionals of M-estimands, we characterize the efficient influence function and construct efficient autoDML estimators via one-step correction, targeted minimum loss estimation, and sieve-based plug-in methods. Under quadratic risk, these estimators exhibit double robustness for linear functionals. We further show they are insensitive to mild misspecification of the M-estimand model, incurring only second-order bias. We illustrate the method by estimating long-term survival probabilities under a semiparametric beta-geometric model.

Automatic Debiased Machine Learning for Smooth Functionals of Nonparametric M-Estimands

TL;DR

This work introduces automatic debiased machine learning (autoDML) for inference on smooth functionals of nonparametric M-estimands, unifying and automating debiasing through Neyman orthogonality and a Hessian-induced Riesz representer. By reducing inference to two learning tasks—estimating the M-estimand and the Hessian representer—the framework yields estimators (one-step, TMLE, sieve) that are regular, asymptotically linear, and, under suitable conditions, semiparametrically efficient for the target functional. A functional von Mises expansion characterizes the efficient influence function as , with the Hessian Riesz representer identified via a risk-minimization problem; this underpins automatic bias corrections and confidence interval construction. The approach extends existing autoDML beyond regression functionals to functionals of vector-valued M-estimands and functionals that depend on nuisance components, achieving robustness to mild misspecification and enabling adaptive model selection through autoSieve. Empirical illustration on a beta-geometric survival model shows favorable finite-sample performance, highlighting the practical impact for complex semiparametric inference in causal settings and survival analysis.

Abstract

We develop a unified framework for automatic debiased machine learning (autoDML) to simplify inference for a broad class of statistical parameters. It applies to any smooth functional of a nonparametric \emph{M-estimand}, defined as the minimizer of a population risk over an infinite-dimensional linear space. Examples of M-estimands include counterfactual regression, quantile, and survival functions, as well as conditional average treatment effects. Rather than requiring manual derivation of influence functions, the framework automates the construction of debiased estimators using three components: the gradient and Hessian of the loss function and a linear approximation of the target functional. Estimation reduces to solving two risk minimization problems -- one for the M-estimand and one for a Riesz representer. The framework accommodates Neyman-orthogonal loss functions depending on nuisance parameters and extends to vector-valued M-estimands through joint risk minimization. For functionals of M-estimands, we characterize the efficient influence function and construct efficient autoDML estimators via one-step correction, targeted minimum loss estimation, and sieve-based plug-in methods. Under quadratic risk, these estimators exhibit double robustness for linear functionals. We further show they are insensitive to mild misspecification of the M-estimand model, incurring only second-order bias. We illustrate the method by estimating long-term survival probabilities under a semiparametric beta-geometric model.
Paper Structure (40 sections, 14 theorems, 145 equations, 3 figures, 3 algorithms)

This paper contains 40 sections, 14 theorems, 145 equations, 3 figures, 3 algorithms.

Table of Contents

  1. Introduction
  2. Contributions of this work
  3. Overview of the generalized autoDML framework
  4. Proposed Approach
  5. Connection with classical one-step estimator in parametric models
  6. Examples
  7. Related work
  8. Functional bias expansion and statistical efficiency
  9. Differentiability conditions for loss functions and target parameters
  10. Functional von Mises expansion and efficient influence function
  11. Automatic debiased machine learning
  12. Proposed estimators
  13. Cross-fitting and algorithms for autoDML
  14. Theoretical results
  15. Asymptotic theory for autoDML
  16. ...and 25 more sections

Key Result

Lemma 1

Suppose $\ell_{\eta}(\theta,z) = l\bigl(\theta(z), \eta(z), z\bigr)$, where $l : \mathbb{R}^{d_1} \times \mathbb{R}^{d_2} \times \mathbb{R}^d \to \mathbb{R}$, and let $\mathcal{C}$ denote the closure of the set $\{(a, b, z) : z \in \mathcal{Z},\, \theta \in \mathcal{H}_{\mathcal{P}},\, \eta \in \mat Moreover, assume that $l$, $\partial_a l$, $\partial_a^2 l$, and $\partial_a \partial_b l$ are join

Figures (3)

  • Figure 1: Given a specification of the orthogonal loss function and the target functional of the M-estimand, the diagram illustrates the autoDML workflow.
  • Figure 2: Performance metrics as a function of sample size ($n$) for different estimators: (Left) Absolute Bias, (Middle) Standard Error, and (Right) Coverage (95%).
  • Figure 3: Bias, standard error, and coverage probability for the ATE, comparing various methods across sample sizes.

Theorems & Definitions (40)

  • Example 1a: Functionals of regressions and representers
  • Example 1b: Functionals in semiparametric regression models
  • Example 1c: Functionals of pseudo-outcome regressions
  • Lemma 1
  • Example 1a: continued
  • Theorem 1: Functional von Mises expansion
  • Theorem 2: Pathwise differentiability
  • Example 1a: continued
  • Example 1a: continued
  • Theorem 3
  • ...and 30 more