Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Estimation and Inference in Conditional Moment Models via the Discrepancy Principle

Jiyuan Tan, Vasilis Syrgkanis

TL;DR

A discrepancy-principle-based framework for adaptive hyperparameter selection that automatically balances bias and variance without relying on the unknown smoothness parameter is introduced, providing a practical, theoretically grounded approach for adaptive inference in ill-posed econometric models.

Abstract

We study adaptive estimation and inference in ill-posed linear inverse problems defined by conditional moment restrictions. Existing regularized estimators such as Regularized DeepIV (RDIV) require prior knowledge of the smoothness of the nuisance function, typically encoded by a beta source condition to tune their regularization parameters. In practice, this smoothness is unknown, and misspecified hyperparameters can lead to suboptimal convergence or instability. We introduce a discrepancy-principle-based framework for adaptive hyperparameter selection that automatically balances bias and variance without relying on the unknown smoothness parameter. Our framework applies to both RDIV (Li et al. [2024]) and the Tikhonov Regularized Adversarial Estimator (TRAE) (Bennett et al. [2023a]) and achieves the same rates in both weak and strong metrics. Building on this, we construct a fully adaptive doubly robust estimator for linear functionals that attains the optimal rate of the better-conditioned primal or dual problem, providing a practical, theoretically grounded approach for adaptive inference in ill-posed econometric models.

Adaptive Estimation and Inference in Conditional Moment Models via the Discrepancy Principle

TL;DR

A discrepancy-principle-based framework for adaptive hyperparameter selection that automatically balances bias and variance without relying on the unknown smoothness parameter is introduced, providing a practical, theoretically grounded approach for adaptive inference in ill-posed econometric models.

Abstract

We study adaptive estimation and inference in ill-posed linear inverse problems defined by conditional moment restrictions. Existing regularized estimators such as Regularized DeepIV (RDIV) require prior knowledge of the smoothness of the nuisance function, typically encoded by a beta source condition to tune their regularization parameters. In practice, this smoothness is unknown, and misspecified hyperparameters can lead to suboptimal convergence or instability. We introduce a discrepancy-principle-based framework for adaptive hyperparameter selection that automatically balances bias and variance without relying on the unknown smoothness parameter. Our framework applies to both RDIV (Li et al. [2024]) and the Tikhonov Regularized Adversarial Estimator (TRAE) (Bennett et al. [2023a]) and achieves the same rates in both weak and strong metrics. Building on this, we construct a fully adaptive doubly robust estimator for linear functionals that attains the optimal rate of the better-conditioned primal or dual problem, providing a practical, theoretically grounded approach for adaptive inference in ill-posed econometric models.
Paper Structure (17 sections, 20 theorems, 162 equations, 3 figures, 1 algorithm)

This paper contains 17 sections, 20 theorems, 162 equations, 3 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Nonparametric Instrumental Variable.
  4. Inverse Problem
  5. Preliminary
  6. Discrepancy‐Principle‐Based Adaptive Regularization
  7. Adaptive Regularized DeepIV Estimator
  8. Adaptive Tikhonov Regularized Adversarial Estimator
  9. Application for Double Robust Estimation
  10. Experiments
  11. Experiment design.
  12. Results
  13. Conclusion
  14. Proofs of \ref{['sec:rdiv']}
  15. Proofs of \ref{['sec:trae']}
  16. ...and 2 more sections

Key Result

Proposition 3.4

Suppose that Assumptions asp:range, asp:source_condition, asp:closeness_deepiv, asp:re_cd, and asp:cr_deepiv hold. Then, there exist sufficiently large constants $c_d, N > 0$ such that for all $n > N$, the output of alg:discrepancy---given inputs $\lambda_0 = 2$ and $\delta = c_d \delta_n$, where $\

Figures (3)

  • Figure 1: Mean absolute error of the DeepIV estimator using different regularization parameters. Each experiment is repeated 50 times.
  • Figure 2: (a) Mean absolute error of the TRAE primal estimator using different regularization parameters.
  • Figure 3: (b) Mean absolute error of the TRAE double robust estimator using different regularization parameters.

Theorems & Definitions (22)

  • Definition 2.2: Discrepancy Principle
  • Proposition 3.4
  • Theorem 3.5
  • Lemma 3.6
  • Lemma 3.7
  • Remark 3.8
  • Proposition 3.12
  • Theorem 3.13
  • Corollary 3.14
  • Lemma A.1
  • ...and 12 more