Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Asymptotics for M-type smoothing splines with non-smooth objective functions

Ioannis Kalogridis

TL;DR

This work extends the asymptotic theory of smoothing splines to M-type estimators with non-smooth convex losses, showing that optimal convergence rates for the regression function and its derivatives hold under mild, moment-free conditions and with possible auxiliary scale estimation. The authors develop an RKHS-based framework to prove existence and derive rates, establish efficient computation via a B-spline representation and iterative reweighting, and propose robust smoothing-parameter selection through a generalized cross-validation criterion. Through simulations and a real-data example on urban air pollution, the paper demonstrates that non-smooth M-type splines rival least-squares splines on regular data and outperform LS splines in the presence of outliers and anomalies. The results support the practical utility of robust smoothing splines and point to extensions such as random-design and dependent-error scenarios.

Abstract

M-type smoothing splines are a broad class of spline estimators that include the popular least-squares smoothing spline but also spline estimators that are less susceptible to outlying observations and model-misspecification. However, available asymptotic theory only covers smoothing spline estimators based on smooth objective functions and consequently leaves out frequently used resistant estimators such as quantile and Huber-type smoothing splines. We provide a general treatment in this paper and, assuming only the convexity of the objective function, show that the least-squares (super-)convergence rates can be extended to M-type estimators whose asymptotic properties have not been hitherto described. We further show that auxiliary scale estimates may be handled under significantly weaker assumptions than those found in the literature and we establish optimal rates of convergence for the derivatives, which have not been obtained outside the least-squares framework. A simulation study and a real-data example illustrate the competitive performance of non-smooth M-type splines in relation to the least-squares spline on regular data and their superior performance on data that contain anomalies.

Asymptotics for M-type smoothing splines with non-smooth objective functions

TL;DR

This work extends the asymptotic theory of smoothing splines to M-type estimators with non-smooth convex losses, showing that optimal convergence rates for the regression function and its derivatives hold under mild, moment-free conditions and with possible auxiliary scale estimation. The authors develop an RKHS-based framework to prove existence and derive rates, establish efficient computation via a B-spline representation and iterative reweighting, and propose robust smoothing-parameter selection through a generalized cross-validation criterion. Through simulations and a real-data example on urban air pollution, the paper demonstrates that non-smooth M-type splines rival least-squares splines on regular data and outperform LS splines in the presence of outliers and anomalies. The results support the practical utility of robust smoothing splines and point to extensions such as random-design and dependent-error scenarios.

Abstract

M-type smoothing splines are a broad class of spline estimators that include the popular least-squares smoothing spline but also spline estimators that are less susceptible to outlying observations and model-misspecification. However, available asymptotic theory only covers smoothing spline estimators based on smooth objective functions and consequently leaves out frequently used resistant estimators such as quantile and Huber-type smoothing splines. We provide a general treatment in this paper and, assuming only the convexity of the objective function, show that the least-squares (super-)convergence rates can be extended to M-type estimators whose asymptotic properties have not been hitherto described. We further show that auxiliary scale estimates may be handled under significantly weaker assumptions than those found in the literature and we establish optimal rates of convergence for the derivatives, which have not been obtained outside the least-squares framework. A simulation study and a real-data example illustrate the competitive performance of non-smooth M-type splines in relation to the least-squares spline on regular data and their superior performance on data that contain anomalies.

Paper Structure

This paper contains 6 sections, 6 theorems, 42 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Main results: existence of solutions and rates of convergence
  3. Computation and smoothing parameter selection
  4. A Monte-Carlo study
  5. Real data example: urban air pollution in Italy
  6. Concluding remarks

Key Result

Theorem 1

If $\rho(x)$ is a lower semicontinuous, nondecreasing unbounded function of $|x|$ and $n \geq m$, the minimization problem has a solution in $\mathcal{W}^{m,2}([0,1])$.

Figures (2)

  • Figure 1: Left: scatter plot of benzene versus temperature. Right: scatter plot of nitrogen oxide versus temperature. The lines (, ) correspond to the Huber and least-squares estimators respectively, while the symbols ($\bullet$${\textcolor{TS}{\blacklozenge}}$$\blacktriangle$) denote Huber weights in $(0.66, 1] ,\ (0.33, 0.66]\ \text{and}\ (0, 0.33]$ respectively.
  • Figure 2: Left: scatter plot of benzene versus temperature. Right: scatter plot of nitrogen oxide versus temperature. The lines (, , , , ) correspond to estimated $(0.1, \ 0.3, \ 0.5, \ 0.7, \ 0.9)$-quantiles respectively.

Theorems & Definitions (6)

  • Theorem 1
  • Theorem 2
  • Corollary 1
  • Corollary 2
  • Theorem 3
  • Theorem 4