Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Local Polynomial Lp-norm Regression

Ladan Tazik, James Stafford, John Braun

TL;DR

This work addresses regression under non-Gaussian noise by proposing local polynomial $L_p$-norm regression, replacing the traditional local least squares with a weighted $L_p$ objective under a GED error model. It develops local constant and local linear estimators, derives their asymptotic bias and variance, and provides a data-driven approach to select the shape parameter $p$ via a robust $Q$ method, along with an effective bandwidth rule $h_p$. Theoretical results show asymptotic normality and comparable bias to $L_2$ methods while adapting variance to error moments; simulations and real-data applications demonstrate superior performance of the LLP approach, with promising 2D extensions. The method is complemented by bootstrap confidence bands and practical algorithms, making it a robust alternative for non-Gaussian settings with potentially heavy tails or outliers.

Abstract

The local least squares estimator for a regression curve cannot provide optimal results when non-Gaussian noise is present. Both theoretical and empirical evidence suggests that residuals often exhibit distributional properties different from those of a normal distribution, making it worthwhile to consider estimation based on other norms. It is suggested that $L_p$-norm estimators be used to minimize the residuals when these exhibit non-normal kurtosis. In this paper, we propose a local polynomial $L_p$-norm regression that replaces weighted least squares estimation with weighted $L_p$-norm estimation for fitting the polynomial locally. We also introduce a new method for estimating the parameter $p$ from the residuals, enhancing the adaptability of the approach. Through numerical and theoretical investigation, we demonstrate our method's superiority over local least squares in one-dimensional data and show promising outcomes for higher dimensions, specifically in 2D.

Local Polynomial Lp-norm Regression

TL;DR

This work addresses regression under non-Gaussian noise by proposing local polynomial -norm regression, replacing the traditional local least squares with a weighted objective under a GED error model. It develops local constant and local linear estimators, derives their asymptotic bias and variance, and provides a data-driven approach to select the shape parameter via a robust method, along with an effective bandwidth rule . Theoretical results show asymptotic normality and comparable bias to methods while adapting variance to error moments; simulations and real-data applications demonstrate superior performance of the LLP approach, with promising 2D extensions. The method is complemented by bootstrap confidence bands and practical algorithms, making it a robust alternative for non-Gaussian settings with potentially heavy tails or outliers.

Abstract

The local least squares estimator for a regression curve cannot provide optimal results when non-Gaussian noise is present. Both theoretical and empirical evidence suggests that residuals often exhibit distributional properties different from those of a normal distribution, making it worthwhile to consider estimation based on other norms. It is suggested that -norm estimators be used to minimize the residuals when these exhibit non-normal kurtosis. In this paper, we propose a local polynomial -norm regression that replaces weighted least squares estimation with weighted -norm estimation for fitting the polynomial locally. We also introduce a new method for estimating the parameter from the residuals, enhancing the adaptability of the approach. Through numerical and theoretical investigation, we demonstrate our method's superiority over local least squares in one-dimensional data and show promising outcomes for higher dimensions, specifically in 2D.

Paper Structure

This paper contains 16 sections, 2 theorems, 20 equations, 3 figures, 18 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Local Polynomial $L_p$Norm Regression
  3. Theoretical Properties
  4. Bandwidth Selection
  5. Estimating P
  6. Simulation Studies and Examples
  7. Simulation Studies
  8. Results
  9. Application to Real Life Data
  10. Bootstrap Confidence Bands
  11. Conclusion and Future Directions
  12. Algorithms
  13. Tables
  14. Triangular Dist. ($-0.5,0.5$)
  15. Bimodal Dist.
  16. ...and 1 more sections

Key Result

Lemma 3.1

Assume A.2 through A.5. For any $x \in (a+h, b-h)$, and for $j=0,1$, define Then

Figures (3)

  • Figure 1: Comparison of the estimated $p$ values by two methods. The results are obtained by averaging 1000 simulations for sample size 100, for varying $p \in [1,20]$.
  • Figure 2: Comparison of average MSEs of LLS and LLP for GED error distribution ($\mu$=0, $\sigma_p$ = 0.2). Plots (a) to (d) correspond to function 1 to 4, respectively. Solid lines represent local least squares estimator (LLS) and dashed lines show local linear $L_p$-norm estimator (LLP).
  • Figure 3: (a) Comparison of fitted line by Local Least Squares and Local Linear $L_p$ norm estimator with the confidence band for Beluga dataset. (b) Comparison of absolute error between Local Linear $L_p$ norm estimator (solid line) and Local Least Squares (dotted line) for Rectangles dataset.

Theorems & Definitions (2)

  • Lemma 3.1
  • Theorem 3.2