Nonparametric Instrumental Variable Regression with Observed Covariates

TL;DR

This work introduces NPIV-O, a nonparametric instrumental-variable regression framework that incorporates observed covariates O to identify heterogeneous causal effects in Y = f_*(X,O) + ε with E[ε|Z,O]=0. It develops the KIV-O algorithm, a two-stage RKHS-based method that first learns a conditional-mean embedding via a vector-valued kernel and then performs a second-stage kernel regression with anisotropic Gaussian kernels tuned to the intrinsic smoothness of f_*, capturing mixed-smoothness in the X and O directions. The authors establish an upper L^2-learning-rate for KIV-O and the first L^2-minimax lower bound for NPIV-O, showing rates that interpolate between NPIV and NPR and demonstrating adaptivity to anisotropic Besov smoothness; they also reveal a gap between upper and lower bounds attributable to kernel-lengthscale choices and partial smoothing, with connections to proximal causal inference. Overall, the paper advances the theory and methodology for NPIV-O, offering a principled way to leverage observed covariates for heterogeneity-aware causal estimation while clarifying fundamental limits and practical considerations for kernel-based operators. The results have potential impact on causal inference in settings with high-dimensional covariates and partially identified models, including proximal causal learning contexts.

Abstract

We study the problem of nonparametric instrumental variable regression with observed covariates, which we refer to as NPIV-O. Compared with standard nonparametric instrumental variable regression (NPIV), the additional observed covariates facilitate causal identification and enables heterogeneous causal effect estimation. However, the presence of observed covariates introduces two challenges for its theoretical analysis. First, it induces a partial identity structure, which renders previous NPIV analyses - based on measures of ill-posedness, stability conditions, or link conditions - inapplicable. Second, it imposes anisotropic smoothness on the structural function. To address the first challenge, we introduce a novel Fourier measure of partial smoothing; for the second challenge, we extend the existing kernel 2SLS instrumental variable algorithm with observed covariates, termed KIV-O, to incorporate Gaussian kernel lengthscales adaptive to the anisotropic smoothness. We prove upper $L^2$-learning rates for KIV-O and the first $L^2$-minimax lower learning rates for NPIV-O. Both rates interpolate between known optimal rates of NPIV and nonparametric regression (NPR). Interestingly, we identify a gap between our upper and lower bounds, which arises from the choice of kernel lengthscales tuned to minimize a projected risk. Our theoretical analysis also applies to proximal causal inference, an emerging framework for causal effect estimation that shares the same conditional moment restriction as NPIV-O.

Figures (5)

• Figure 1: Illustration for LF($\gamma$) and HF($\gamma$).
• Figure 2: This figure illustrates the partial Fourier transform of $f_\mathbf v$ in Eq. \ref{['eq:mathfrak_F']}, where we take $d_x = 1$, $\mathfrak{K} = 2$, $\mathfrak{m} = 4$, $s_x = \underline{s} \leq s_o$. The dashed blue line represents the Fourier transform of the B-spline $M_{\mathfrak{K},-2}$. The red line shows the result of applying frequency masks to this B-spline: $\sum_{{\boldsymbol\ell}_x}\beta_{\mathbf v({\boldsymbol\ell}_x,{\boldsymbol\ell}_o)} \mathcal{F}[M_{\mathfrak{K},-2}](\omega_x) \cdot \mathbbm{1}_{{\boldsymbol\ell}_x}(2^{-\mathfrak{K}}\omega_x)$. The Fourier transform of $f_\mathbf v(\cdot, \mathbf o)$ is equal to this result up to scaling factors dependent only on $\mathbf o$.
• Figure 3: Roadmap for proof of \ref{['prop:projected_stage_one']}.
• Figure 4: Roadmap for proof of \ref{['prop: projected upper rate']}.
• Figure 5: Roadmap for proof of \ref{['thm:upper_rate']}.

Theorems & Definitions (103)

• Definition 1: Vector-valued dominating mixed-smoothness Sobolev space
• Remark 2.1: Choice of Stage I kernels $k_Z, k_{O,1}$
• Definition 2: Modulus of smoothness
• Definition 3: Anisotropic Besov space $B_{p, q}^{\boldsymbol{s}}(\nu)$
• Remark 2.2: Choice of Stage II kernels $k_{X}, k_{O,2}$
• Remark 2.3: Why use different kernels in Stage I and Stage II
• Remark 4.1: Connection with sieve measure of ill-posedness
• Remark 4.2: Existing treatment of partial smoothing of $T$ in NPIV-O
• Theorem 4.1: Upper learning rate for KIV-O
• Remark 4.3