Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sharp Structure-Agnostic Lower Bounds for General Functional Estimation

Jikai Jin, Vasilis Syrgkanis

TL;DR

<3-5 sentence high-level summary> This paper develops sharp minimax lower bounds for a broad class of semiparametric functionals that depend on unknown nuisance functions via generalized regression. Using a novel two-step perturbation scheme and the method of fuzzy hypotheses, the authors characterize two regimes: an affine/mixed-bias regime with optimal rate $\Omega(\epsilon_{n,\gamma}\epsilon_{n,\alpha}+n^{-1/2})$, and a general non-affine regime that incurs an additional $\Omega(\epsilon_{n,\gamma}^2)$ term, with first-order debiasing methods (DML) achieving matching upper bounds in both. They instantiate the results for estimands including ATE, ECC, WAD, APE, DS, LOD, and EQD, providing theoretical validation for widely used first-order debiasing procedures in structure-agnostic settings. The findings guide practitioners on the limits of nuisance-robust estimation without structural assumptions and illuminate when curvature effects fundamentally constrain performance.

Abstract

The design of efficient nonparametric estimators has long been a central problem in statistics, machine learning, and decision making. Classical optimal procedures often rely on strong structural assumptions, which can be misspecified in practice and complicate deployment. This limitation has sparked growing interest in structure-agnostic approaches -- methods that debias black-box nuisance estimates without imposing structural priors. Understanding the fundamental limits of these methods is therefore crucial. This paper provides a systematic investigation of the optimal error rates achievable by structure-agnostic estimators. We first show that, for estimating the average treatment effect (ATE), a central parameter in causal inference, doubly robust learning attains optimal structure-agnostic error rates. We then extend our analysis to a general class of functionals that depend on unknown nuisance functions and establish the structure-agnostic optimality of debiased/double machine learning (DML). We distinguish two regimes -- one where double robustness is attainable and one where it is not -- leading to different optimal rates for first-order debiasing, and show that DML is optimal in both regimes. Finally, we instantiate our general lower bounds by deriving explicit optimal rates that recover existing results and extend to additional estimands of interest. Our results provide theoretical validation for widely used first-order debiasing methods and guidance for practitioners seeking optimal approaches in the absence of structural assumptions. This paper generalizes and subsumes the ATE lower bound established in \citet{jin2024structure} by the same authors.

Sharp Structure-Agnostic Lower Bounds for General Functional Estimation

TL;DR

<3-5 sentence high-level summary> This paper develops sharp minimax lower bounds for a broad class of semiparametric functionals that depend on unknown nuisance functions via generalized regression. Using a novel two-step perturbation scheme and the method of fuzzy hypotheses, the authors characterize two regimes: an affine/mixed-bias regime with optimal rate , and a general non-affine regime that incurs an additional term, with first-order debiasing methods (DML) achieving matching upper bounds in both. They instantiate the results for estimands including ATE, ECC, WAD, APE, DS, LOD, and EQD, providing theoretical validation for widely used first-order debiasing procedures in structure-agnostic settings. The findings guide practitioners on the limits of nuisance-robust estimation without structural assumptions and illuminate when curvature effects fundamentally constrain performance.

Abstract

The design of efficient nonparametric estimators has long been a central problem in statistics, machine learning, and decision making. Classical optimal procedures often rely on strong structural assumptions, which can be misspecified in practice and complicate deployment. This limitation has sparked growing interest in structure-agnostic approaches -- methods that debias black-box nuisance estimates without imposing structural priors. Understanding the fundamental limits of these methods is therefore crucial. This paper provides a systematic investigation of the optimal error rates achievable by structure-agnostic estimators. We first show that, for estimating the average treatment effect (ATE), a central parameter in causal inference, doubly robust learning attains optimal structure-agnostic error rates. We then extend our analysis to a general class of functionals that depend on unknown nuisance functions and establish the structure-agnostic optimality of debiased/double machine learning (DML). We distinguish two regimes -- one where double robustness is attainable and one where it is not -- leading to different optimal rates for first-order debiasing, and show that DML is optimal in both regimes. Finally, we instantiate our general lower bounds by deriving explicit optimal rates that recover existing results and extend to additional estimands of interest. Our results provide theoretical validation for widely used first-order debiasing methods and guidance for practitioners seeking optimal approaches in the absence of structural assumptions. This paper generalizes and subsumes the ATE lower bound established in \citet{jin2024structure} by the same authors.

Paper Structure

This paper contains 122 sections, 41 theorems, 442 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Contributions and main message.
  3. Technical contributions.
  4. Background
  5. Notation
  6. Overview of our contributions
  7. Our main results in a nutshell
  8. Our main technical contribution
  9. Optimality of first-order debiasing: the ATE case
  10. General error rates of first-order debiasing estimators
  11. Riesz representation for linear functionals.
  12. Generalized regression for $\gamma$ and the score $\rho$.
  13. The weighted Riesz identity and the auxiliary nuisance $\alpha$.
  14. The first-order debiased estimator.
  15. The doubly robust regime and the mixed bias property
  16. ...and 107 more sections

Key Result

Theorem 3.1

Under certain assumptions that we verify for a broad class of functionals, the optimal worst-case error for estimating $\theta$ in eq:parameter-of-interest is either $\Omega({\epsilon}_{n,\gamma}{\epsilon}_{n,\alpha}+n^{-1/2})$ or $\Omega({\epsilon}_{n,\gamma}{\epsilon}_{n,\alpha}+{\epsilon}_{n,\gam

Figures (1)

  • Figure 1: Schematic view of the anchored analysis. In general the first-stage nuisance estimates $\hat{h}$ (think $(\hat{\gamma},\hat{\alpha})$) need not be exactly induced by any feasible distribution. Lemma \ref{['lemma:restriction-to-feasible-space']} shows that for lower bounds it is enough to work in an anchored neighborhood around a feasible $\hat{P}$ whose induced nuisances are within the same error tolerances. The blue intersection represents the uncertainty set of distributions compatible with the nuisance-error constraints.

Theorems & Definitions (49)

  • Theorem 3.1: Informal minimax structure-agnostic rates
  • Theorem 4.1: Doubly robust ATE upper bound
  • Theorem 4.2: Minimax lower bound for ATE
  • Remark 4.1
  • Theorem 5.1: Generic first-order debiasing upper bound (variant of chernozhukov2021automatic, Theorem 3.3)
  • Lemma 6.1: Anchoring to a feasible nuisance pair
  • Remark 6.1
  • Definition 6.1: Feasible distributions
  • Definition 6.2: Nondegenerate measure space
  • Definition 6.3: Feasible perturbations
  • ...and 39 more