Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Convex Regression in Multidimensions: Suboptimality of Least Squares Estimators

Gil Kur, Fuchang Gao, Adityanand Guntuboyina, Bodhisattva Sen

Abstract

Under the usual nonparametric regression model with Gaussian errors, Least Squares Estimators (LSEs) over natural subclasses of convex functions are shown to be suboptimal for estimating a $d$-dimensional convex function in squared error loss when the dimension $d$ is 5 or larger. The specific function classes considered include: (i) bounded convex functions supported on a polytope (in random design), (ii) Lipschitz convex functions supported on any convex domain (in random design), (iii) convex functions supported on a polytope (in fixed design). For each of these classes, the risk of the LSE is proved to be of the order $n^{-2/d}$ (up to logarithmic factors) while the minimax risk is $n^{-4/(d+4)}$, when $d \ge 5$. In addition, the first rate of convergence results (worst case and adaptive) for the unrestricted convex LSE are established in fixed-design for polytopal domains for all $d \geq 1$. Some new metric entropy results for convex functions are also proved which are of independent interest.

Convex Regression in Multidimensions: Suboptimality of Least Squares Estimators

Abstract

Under the usual nonparametric regression model with Gaussian errors, Least Squares Estimators (LSEs) over natural subclasses of convex functions are shown to be suboptimal for estimating a -dimensional convex function in squared error loss when the dimension is 5 or larger. The specific function classes considered include: (i) bounded convex functions supported on a polytope (in random design), (ii) Lipschitz convex functions supported on any convex domain (in random design), (iii) convex functions supported on a polytope (in fixed design). For each of these classes, the risk of the LSE is proved to be of the order (up to logarithmic factors) while the minimax risk is , when . In addition, the first rate of convergence results (worst case and adaptive) for the unrestricted convex LSE are established in fixed-design for polytopal domains for all . Some new metric entropy results for convex functions are also proved which are of independent interest.

Paper Structure

This paper contains 41 sections, 33 theorems, 352 equations, 6 tables.

Table of Contents

  1. Introduction
  2. Fixed-design results for the unrestricted convex LSE
  3. Paper Outline
  4. A note on constants underlying our rate results
  5. Minimax Rates for Convex Regression
  6. Random Design
  7. Fixed Design
  8. LSE Rates for Convex Regression
  9. Random Design
  10. Results for the unrestricted convex LSE in fixed design
  11. Proof Sketches for Results in Section \ref{['LseRates']}
  12. General LSE Accuracy result
  13. High-level intuition for the proof of Theorem \ref{['implb.blse']}
  14. Proof Sketch for Theorem \ref{['implb.blse']}
  15. Proof ideas for Lemma \ref{['GrUB']} and Lemma \ref{['GrLB']}
  16. ...and 26 more sections

Key Result

Proposition 2.1

For every $\Omega$ satisfying OmegaAss, we have where $c_{d, L, \sigma}$ and $C_{d, L, \sigma}$ are positive constants depending only on $d, L, \sigma$.

Theorems & Definitions (62)

  • Proposition 2.1
  • Proposition 2.2
  • Proposition 2.3
  • Theorem 3.1
  • Lemma 3.2
  • Theorem 3.3
  • Theorem 3.4
  • Theorem 3.5
  • Theorem 3.6
  • Theorem 4.1: Chatterjee
  • ...and 52 more