Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Efficient Minimax Optimal Estimator For Multivariate Convex Regression

Gil Kur, Eli Putterman

TL;DR

The paper addresses minimax-optimal estimation in high-dimensional multivariate convex regression for $d\ge 5$ under $L$-Lipschitz and $\Gamma$-bounded constraints. It introduces a novel estimation strategy based on a $k(n)$-simplicial convex approximation, data-driven triangulation, per-simplex affine regression, and a specialized norm-estimation mechanism grounded in potential theory. A convex optimization step enforces $1$-Lipschitz constraints and, via a projection MP, yields a proper estimator with runtime $n^{O(d)}$ and risk $\tilde{O}_d(n^{-\frac{4}{d+4}})$. The approach achieves minimax optimality up to polylog factors, beating the suboptimal LSE in the non-Donsker regime, and provides a conceptual framework for scalable estimators in non-Donsker convex regression. These results broaden the scope of computationally feasible, statistically optimal procedures for high-dimensional, shape-constrained regression and suggest practical directions for scalable implementations.

Abstract

This work studies the computational aspects of multivariate convex regression in dimensions $d \ge 5$. Our results include the \emph{first} estimators that are minimax optimal (up to logarithmic factors) with polynomial runtime in the sample size for both $L$-Lipschitz convex regression, and $Γ$-bounded convex regression under polytopal support. Our analysis combines techniques from empirical process theory, stochastic geometry, and potential theory, and leverages recent algorithmic advances in mean estimation for random vectors and in distribution-free linear regression. These results provide the first efficient, minimax-optimal procedures for non-Donsker classes for which their corresponding least-squares estimator is provably minimax-suboptimal.

An Efficient Minimax Optimal Estimator For Multivariate Convex Regression

TL;DR

The paper addresses minimax-optimal estimation in high-dimensional multivariate convex regression for under -Lipschitz and -bounded constraints. It introduces a novel estimation strategy based on a -simplicial convex approximation, data-driven triangulation, per-simplex affine regression, and a specialized norm-estimation mechanism grounded in potential theory. A convex optimization step enforces -Lipschitz constraints and, via a projection MP, yields a proper estimator with runtime and risk . The approach achieves minimax optimality up to polylog factors, beating the suboptimal LSE in the non-Donsker regime, and provides a conceptual framework for scalable estimators in non-Donsker convex regression. These results broaden the scope of computationally feasible, statistically optimal procedures for high-dimensional, shape-constrained regression and suggest practical directions for scalable implementations.

Abstract

This work studies the computational aspects of multivariate convex regression in dimensions . Our results include the \emph{first} estimators that are minimax optimal (up to logarithmic factors) with polynomial runtime in the sample size for both -Lipschitz convex regression, and -bounded convex regression under polytopal support. Our analysis combines techniques from empirical process theory, stochastic geometry, and potential theory, and leverages recent algorithmic advances in mean estimation for random vectors and in distribution-free linear regression. These results provide the first efficient, minimax-optimal procedures for non-Donsker classes for which their corresponding least-squares estimator is provably minimax-suboptimal.
Paper Structure (26 sections, 23 theorems, 124 equations, 1 algorithm)

This paper contains 26 sections, 23 theorems, 124 equations, 1 algorithm.

Table of Contents

  1. Introduction and Main Results
  2. Proposed Estimator of Theorem \ref{['Thm:ConvexFunc']}
  3. Notations and preliminaries
  4. Proof of Theorem \ref{['Thm:ConvexFunc']}
  5. On Lemma \ref{['Lem:VerMSE']}: Optimal estimation of a norm of a convex function
  6. On Lemma \ref{['Lem:LoneNorm']}
  7. Definitions and Preliminaries
  8. Basic notions regarding polytopes
  9. Proofs of Missing Parts
  10. Proof of Theorem \ref{['Cor:Aproximation']}
  11. $f_k$ is $k$-simplicial with probability at least $9/10$:
  12. Bounding $\mathbb{P}(\Omega \backslash \Omega_k)$ with probability at least $9/10$:
  13. Bounding $\int (f - f_k)^2d\mathbb{P}$ with probability at least $9/10$:
  14. Deriving the theorem
  15. Proof of Lemma \ref{['Lem:RandomSimp']}
  16. ...and 11 more sections

Key Result

Theorem 2

Let $d \geq 5$ and $n \geq d+1$. Then, under Assumption Ass:One, in the task of $L$-Lipschitz convex regression on a convex domain $\Omega \subset B_d$, there exists an estimator, $\widehat{f}_{L}$, with runtime of at most $n^{O(d)}$ such that where $h(d) \leq 3d$.

Theorems & Definitions (36)

  • Theorem 2
  • Theorem 3
  • Remark 4
  • Remark 5
  • Definition 6
  • Definition 7
  • Lemma 8
  • Lemma 9
  • Theorem 10
  • Lemma 11
  • ...and 26 more