Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A theory of shape regularity for local regression maps

Jérémy Bettinger, François Portier, Adrien Saumard

TL;DR

This work develops a unifying theory of shape-regular regression maps to obtain sharp pointwise and uniform convergence rates for a broad class of local regression estimators, including data-dependent partitions like CART-like trees and purely random trees. By combining VC theory with a minimal-mass condition and a geometric shape-regularity constraint, it proves that optimal rates for Lipschitz regression functions are both necessary and sufficient, and derives explicit deviation bounds for NN, CART-like trees, and Mondrian-type partitions. The results clarify when common local-regression schemes attain the minimax rate $n^{-1/(d+2)}$ (up to logs) and reveal which tree constructions fail to be SR and thus fail to achieve optimal rates. Practically, the framework provides verifiable, geometry-based criteria to guarantee reliable pointwise and uniform performance across a wide range of local regression methods.

Abstract

We introduce the concept of shape-regular regression maps as a framework to derive optimal rates of convergence for various non-parametric local regression estimators. Using Vapnik-Chervonenkis theory, we establish upper and lower bounds on the pointwise and the sup-norm estimation error, even when the localization procedure depends on the full data sample, and under mild conditions on the regression model. Our results demonstrate that the shape regularity of regression maps is not only sufficient but also necessary to achieve an optimal rate of convergence for Lipschitz regression functions. To illustrate the theory, we establish new concentration bounds for many popular local regression methods such as nearest neighbors algorithm, CART-like regression trees and several purely random trees including Mondrian trees.

A theory of shape regularity for local regression maps

TL;DR

This work develops a unifying theory of shape-regular regression maps to obtain sharp pointwise and uniform convergence rates for a broad class of local regression estimators, including data-dependent partitions like CART-like trees and purely random trees. By combining VC theory with a minimal-mass condition and a geometric shape-regularity constraint, it proves that optimal rates for Lipschitz regression functions are both necessary and sufficient, and derives explicit deviation bounds for NN, CART-like trees, and Mondrian-type partitions. The results clarify when common local-regression schemes attain the minimax rate (up to logs) and reveal which tree constructions fail to be SR and thus fail to achieve optimal rates. Practically, the framework provides verifiable, geometry-based criteria to guarantee reliable pointwise and uniform performance across a wide range of local regression methods.

Abstract

We introduce the concept of shape-regular regression maps as a framework to derive optimal rates of convergence for various non-parametric local regression estimators. Using Vapnik-Chervonenkis theory, we establish upper and lower bounds on the pointwise and the sup-norm estimation error, even when the localization procedure depends on the full data sample, and under mild conditions on the regression model. Our results demonstrate that the shape regularity of regression maps is not only sufficient but also necessary to achieve an optimal rate of convergence for Lipschitz regression functions. To illustrate the theory, we establish new concentration bounds for many popular local regression methods such as nearest neighbors algorithm, CART-like regression trees and several purely random trees including Mondrian trees.

Paper Structure

This paper contains 25 sections, 25 theorems, 214 equations, 1 algorithm.

Table of Contents

  1. Introduction
  2. Mathematical background
  3. Regression set-up
  4. Local regression maps
  5. Example 1 (fixed hyper-rectangles partition).
  6. Example 2 (purely random trees).
  7. Example 3 (nearest neighbors regression).
  8. Example 4 (CART-like trees).
  9. A deviation bound for local map estimators
  10. A deviation bound for the variance term
  11. A pointwise error bound
  12. Shape regularity
  13. Minimal mass assumption
  14. Shape-regular sets
  15. Shape regularity of local maps
  16. ...and 10 more sections

Key Result

Theorem 2

Let $n\geq 1$ and $\delta \in (0,1)$. Suppose that cond:epsilon is fulfilled and that $\{\mathcal{V} (x) \, :\, x\in \mathbb R^d\} \subset \mathcal{A}$, a deterministic collection of sets in $\mathbb R^d$. The following inequality holds with probability at least $1 - \delta$,

Theorems & Definitions (33)

  • Definition 1
  • Theorem 2
  • Definition 3
  • Theorem 4
  • Definition 5
  • Theorem 6
  • Definition 7
  • Definition 8
  • Proposition 9
  • Proposition 10
  • ...and 23 more