Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A new approach to locally adaptive polynomial regression

Sabyasachi Chatterjee, Subhajit Goswami, Soumendu Sundar Mukherjee

TL;DR

This work tackles locally adaptive nonparametric regression by introducing LASER, a bandwidth-selection method driven by a local discrepancy measure that mirrors regression-tree splitting criteria. By selecting the largest interval around each point where the discrepancy is below a universal threshold $\lambda$, LASER achieves near-optimal pointwise adaptation to local Hölder regularity with a single tuning parameter, $\lambda \asymp \sigma\sqrt{\log n}$. The authors establish non-asymptotic risk bounds that adapt to the local smoothness $f \in C^{r_0,\alpha_0}$ and provide a unified analysis across degrees $r$, supported by polynomial-structure lemmas. The method is shown to be competitive with, and often superior to, existing locally adaptive approaches, and is implemented in an R package with efficient dyadic variants, offering a new perspective on bandwidth selection and potential extensions to robust losses and higher dimensions.

Abstract

Adaptive bandwidth selection is a fundamental challenge in nonparametric regression. This paper introduces a new bandwidth selection procedure inspired by the optimality criteria for $\ell_0$-penalized regression. Although similar in spirit to Lepski's method and its variants in selecting the largest interval satisfying an admissibility criterion, our approach stems from a distinct philosophy, utilizing criteria based on $\ell_2$-norms of interval projections rather than explicit point and variance estimates. We obtain non-asymptotic risk bounds for the local polynomial regression methods based on our bandwidth selection procedure which adapt (near-)optimally to the local Hölder exponent of the underlying regression function simultaneously at all points in its domain. Furthermore, we show that there is a single ideal choice of a global tuning parameter in each case under which the above-mentioned local adaptivity holds. The optimal risks of our methods derive from the properties of solutions to a new ``bandwidth selection equation'' which is of independent interest. We believe that the principles underlying our approach provide a new perspective to the classical yet ever relevant problem of locally adaptive nonparametric regression.

A new approach to locally adaptive polynomial regression

TL;DR

This work tackles locally adaptive nonparametric regression by introducing LASER, a bandwidth-selection method driven by a local discrepancy measure that mirrors regression-tree splitting criteria. By selecting the largest interval around each point where the discrepancy is below a universal threshold , LASER achieves near-optimal pointwise adaptation to local Hölder regularity with a single tuning parameter, . The authors establish non-asymptotic risk bounds that adapt to the local smoothness and provide a unified analysis across degrees , supported by polynomial-structure lemmas. The method is shown to be competitive with, and often superior to, existing locally adaptive approaches, and is implemented in an R package with efficient dyadic variants, offering a new perspective on bandwidth selection and potential extensions to robust losses and higher dimensions.

Abstract

Adaptive bandwidth selection is a fundamental challenge in nonparametric regression. This paper introduces a new bandwidth selection procedure inspired by the optimality criteria for -penalized regression. Although similar in spirit to Lepski's method and its variants in selecting the largest interval satisfying an admissibility criterion, our approach stems from a distinct philosophy, utilizing criteria based on -norms of interval projections rather than explicit point and variance estimates. We obtain non-asymptotic risk bounds for the local polynomial regression methods based on our bandwidth selection procedure which adapt (near-)optimally to the local Hölder exponent of the underlying regression function simultaneously at all points in its domain. Furthermore, we show that there is a single ideal choice of a global tuning parameter in each case under which the above-mentioned local adaptivity holds. The optimal risks of our methods derive from the properties of solutions to a new ``bandwidth selection equation'' which is of independent interest. We believe that the principles underlying our approach provide a new perspective to the classical yet ever relevant problem of locally adaptive nonparametric regression.
Paper Structure (18 sections, 10 theorems, 91 equations, 4 figures, 3 algorithms)

This paper contains 18 sections, 10 theorems, 91 equations, 4 figures, 3 algorithms.

Table of Contents

  1. Introduction
  2. Nonparametric regression: local adaptivity
  3. From $\ell_0$-penalized regression to bandwidth selection
  4. Our contributions vis-à-vis related works
  5. Outline
  6. Notation and conventions
  7. Description of LASER and risk bounds
  8. Formal description of the method
  9. Connection to Regression Trees
  10. Local adaptivity of LASER in a toy example
  11. Pointwise risk bounds for LASER
  12. Proof of the main result
  13. Some properties of polynomials
  14. Algorithm and simulations
  15. Pseudocode for LASER
  16. ...and 3 more sections

Key Result

theorem 1

Fix a degree $r \in \N$ and let $f : [0, 1] \to \R$. There exist constants $\Cl{C:lambda}$ and $\Cl{C:bnd} = \Cr{C:bnd}(r)$ such that the following holds with high probability for $\lambda = C_1 \sigma \sqrt{\log n}$. Simultaneously for all quadruplets $(i_0, s_0, r_0, \alpha_0)$ where $i_0 \in [n]$ where $\widehat{f}(\frac{i_0}{n}) = \widehat{f}_{\textsf{LASER}{}(r, \lambda)} (\tfrac{i_0}{n})$ is

Figures (4)

  • Figure 1: The Blocks function. We have used LASER with $r = 0$ and $0$-th order Trend Filtering.
  • Figure 2: The Bumps function. We have used LASER with $r = 2$ and $2$-nd order Trend Filtering.
  • Figure 3: The HeaviSine function. We have used LASER with $r = 2$ and $2$-nd order Trend Filtering.
  • Figure 4: The Doppler function. We have used LASER with $r = 2$ and $2$-nd order Trend Filtering.

Theorems & Definitions (21)

  • definition 1: Hölder space
  • theorem 1: Local Adaptivity Result
  • proposition 1
  • lemma 1
  • remark 1
  • proof
  • proof : Proof of Proposition \ref{['prop:noise_threshold_decomp']}
  • lemma 2
  • proof
  • proposition 2
  • ...and 11 more