Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Convergence rate for Nearest Neighbour matching: geometry of the domain and higher-order regularity

Simon Viel, Lionel Truquet, Ikko Yamane

TL;DR

The paper tackles the problem of estimating covariate-shift expectations using $k$-NN matching, focusing on boundary bias and higher-order bias properties. It introduces two geometric conditions on the covariate domain that generalize convexity, enabling a $k^{2/d} n^{-2/d}$ bias rate and, in low dimension ($d\le 4$), parametric-rate convergence with $k=1$, plus semiparametric efficiency for ATE when $d\le 3$. It also extends these results to local-polynomial estimators, provides nonasymptotic bounds supported by numerical experiments, and clarifies the geometry required for these gains. Collectively, the work offers practical, geometry-aware conditions to accelerate NN-matching performance in covariate-shift and causal-inference tasks, including ATE estimation, while presenting a versatile extension to higher-order local polynomials.

Abstract

Estimating some mathematical expectations from partially observed data and in particular missing outcomes is a central problem encountered in numerous fields such as transfer learning, counterfactual analysis or causal inference. Matching estimators, estimators based on k-nearest neighbours, are widely used in this context. It is known that the variance of such estimators can converge to zero at a parametric rate, but their bias can have a slower rate when the dimension of the covariates is larger than 2. This makes analysis of this bias particularly important. In this paper, we provide higher order properties of the bias. In contrast to the existing literature related to this problem, we do not assume that the support of the target distribution of the covariates is strictly included in that of the source, and we analyse two geometric conditions on the support that avoid such boundary bias problems. We show that these conditions are much more general than the usual convex support assumption, leading to an improvement of existing results. Furthermore, we show that the matching estimator studied by Abadie and Imbens (2006) for the average treatment effect can be asymptotically efficient when the dimension of the covariates is less than 4, a result only known in dimension 1.

Convergence rate for Nearest Neighbour matching: geometry of the domain and higher-order regularity

TL;DR

The paper tackles the problem of estimating covariate-shift expectations using -NN matching, focusing on boundary bias and higher-order bias properties. It introduces two geometric conditions on the covariate domain that generalize convexity, enabling a bias rate and, in low dimension (), parametric-rate convergence with , plus semiparametric efficiency for ATE when . It also extends these results to local-polynomial estimators, provides nonasymptotic bounds supported by numerical experiments, and clarifies the geometry required for these gains. Collectively, the work offers practical, geometry-aware conditions to accelerate NN-matching performance in covariate-shift and causal-inference tasks, including ATE estimation, while presenting a versatile extension to higher-order local polynomials.

Abstract

Estimating some mathematical expectations from partially observed data and in particular missing outcomes is a central problem encountered in numerous fields such as transfer learning, counterfactual analysis or causal inference. Matching estimators, estimators based on k-nearest neighbours, are widely used in this context. It is known that the variance of such estimators can converge to zero at a parametric rate, but their bias can have a slower rate when the dimension of the covariates is larger than 2. This makes analysis of this bias particularly important. In this paper, we provide higher order properties of the bias. In contrast to the existing literature related to this problem, we do not assume that the support of the target distribution of the covariates is strictly included in that of the source, and we analyse two geometric conditions on the support that avoid such boundary bias problems. We show that these conditions are much more general than the usual convex support assumption, leading to an improvement of existing results. Furthermore, we show that the matching estimator studied by Abadie and Imbens (2006) for the average treatment effect can be asymptotically efficient when the dimension of the covariates is less than 4, a result only known in dimension 1.
Paper Structure (28 sections, 28 theorems, 205 equations, 4 figures, 1 table)

This paper contains 28 sections, 28 theorems, 205 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Second-order bias property of matching estimators
  3. Notation and assumptions
  4. Rate for the bias
  5. Note
  6. Note.
  7. Note.
  8. Rate for the variance
  9. Note
  10. Application to average treatment effects
  11. Note.
  12. Analysis of two geometric conditions
  13. Extension to local polynomials
  14. Numerical experiments
  15. Methods that we compare
  16. ...and 13 more sections

Key Result

Proposition 1

Under Assumptions cond:reg1 to cond:reg4, for all $\lambda\ge 1$, we have where $C_{\lambda,d,P,h}:=L^\lambda\,2\,\Gamma(2+\floor{\lambda/d})\, (c\underline p\,\lvert V^d \rvert)^{-\lambda/d}$ and $L$ is the Lipschitz constant for $g$. Here, $\Gamma$ denotes Euler's gamma function and $\floor a$ denotes the integer part of $a\in\mathbb R$.

Figures (4)

  • Figure 1: Illustrations of Proposition \ref{['assumption_counter']}
  • Figure 2: Visualization of the data distributions used in the experiments.
  • Figure 3: Results for Setup TN0.5-Cubic.
  • Figure 4: Results for Setup TN0.5-Cubic Reversed.

Theorems & Definitions (45)

  • Proposition 1
  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Corollary 1
  • Theorem 4
  • Corollary 2
  • Proposition 2
  • Theorem 5
  • Proposition 3
  • ...and 35 more