Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning and Testing Convex Functions

Renato Ferreira Pinto, Cassandra Marcussen, Elchanan Mossel, Shivam Nadimpalli

TL;DR

The paper addresses learning and testing of real-valued convex functions on Gaussian space under Lipschitz regularity, framing the problems in the $L^2(\gamma)$ metric to the convex Lipschitz class ${\mathcal C}(L)$. It introduces an agnostic proper learner with sample complexity $n^{O(L^2/\varepsilon^2)}$ and proves a CSQ lower bound of $n^{\mathrm{poly}(L/\varepsilon)}$, as well as a tolerant two-sided tester with the same sample bounds and a one-sided tester with exponential-in-$n$ samples, leveraging Hermite analysis and the Ornstein–Uhlenbeck operator. The approach combines low-degree (polynomial) approximations with convex regression to obtain a proper learner, and uses an empirical convex envelope to realize one-sided testing, including a novel box-restriction technique to transfer Gaussian-space results to finite domains. The results illuminate the computational and statistical landscape for real-valued convexity in high dimensions, offering polynomial-in-$n$ performance under Lipschitzness and outlining clear directions for further tightening lower bounds and runtime gaps. Overall, the work advances the understanding of convex function learning and testing in continuous, high-dimensional settings and establishes a principled framework that connects approximation theory, Hermite analysis, and convex geometry.

Abstract

We consider the problems of \emph{learning} and \emph{testing} real-valued convex functions over Gaussian space. Despite the extensive study of function convexity across mathematics, statistics, and computer science, its learnability and testability have largely been examined only in discrete or restricted settings -- typically with respect to the Hamming distance, which is ill-suited for real-valued functions. In contrast, we study these problems in high dimensions under the standard Gaussian measure, assuming sample access to the function and a mild smoothness condition, namely Lipschitzness. A smoothness assumption is natural and, in fact, necessary even in one dimension: without it, convexity cannot be inferred from finitely many samples. As our main results, we give: - Learning Convex Functions: An agnostic proper learning algorithm for Lipschitz convex functions that achieves error $\varepsilon$ using $n^{O(1/\varepsilon^2)}$ samples, together with a complementary lower bound of $n^{\mathrm{poly}(1/\varepsilon)}$ samples in the \emph{correlational statistical query (CSQ)} model. - Testing Convex Functions: A tolerant (two-sided) tester for convexity of Lipschitz functions with the same sample complexity (as a corollary of our learning result), and a one-sided tester (which never rejects convex functions) using $O(\sqrt{n}/\varepsilon)^n$ samples.

Learning and Testing Convex Functions

TL;DR

The paper addresses learning and testing of real-valued convex functions on Gaussian space under Lipschitz regularity, framing the problems in the metric to the convex Lipschitz class . It introduces an agnostic proper learner with sample complexity and proves a CSQ lower bound of , as well as a tolerant two-sided tester with the same sample bounds and a one-sided tester with exponential-in- samples, leveraging Hermite analysis and the Ornstein–Uhlenbeck operator. The approach combines low-degree (polynomial) approximations with convex regression to obtain a proper learner, and uses an empirical convex envelope to realize one-sided testing, including a novel box-restriction technique to transfer Gaussian-space results to finite domains. The results illuminate the computational and statistical landscape for real-valued convexity in high dimensions, offering polynomial-in- performance under Lipschitzness and outlining clear directions for further tightening lower bounds and runtime gaps. Overall, the work advances the understanding of convex function learning and testing in continuous, high-dimensional settings and establishes a principled framework that connects approximation theory, Hermite analysis, and convex geometry.

Abstract

We consider the problems of \emph{learning} and \emph{testing} real-valued convex functions over Gaussian space. Despite the extensive study of function convexity across mathematics, statistics, and computer science, its learnability and testability have largely been examined only in discrete or restricted settings -- typically with respect to the Hamming distance, which is ill-suited for real-valued functions. In contrast, we study these problems in high dimensions under the standard Gaussian measure, assuming sample access to the function and a mild smoothness condition, namely Lipschitzness. A smoothness assumption is natural and, in fact, necessary even in one dimension: without it, convexity cannot be inferred from finitely many samples. As our main results, we give: - Learning Convex Functions: An agnostic proper learning algorithm for Lipschitz convex functions that achieves error using samples, together with a complementary lower bound of samples in the \emph{correlational statistical query (CSQ)} model. - Testing Convex Functions: A tolerant (two-sided) tester for convexity of Lipschitz functions with the same sample complexity (as a corollary of our learning result), and a one-sided tester (which never rejects convex functions) using samples.

Paper Structure

This paper contains 33 sections, 38 theorems, 140 equations, 2 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Our Results
  3. Agnostic Proper Learning and Sample-Based Testing of Convex Functions
  4. Lower Bounds for Learning Convex Functions
  5. One-Sided Testing of Lipschitz Convex Functions
  6. Related Work
  7. Learning Convex Functions.
  8. Approximation Theory.
  9. Testing Convexity over Discrete Domains.
  10. $L^p$ Testing of Convexity.
  11. Testing Set Convexity.
  12. Discussion
  13. Analogy with Boolean Monotonicity.
  14. Testing Lower Bounds.
  15. Sample vs. Runtime Gaps.
  16. ...and 18 more sections

Key Result

Theorem 0

Let $\varepsilon, L > 0$ and suppose $f : \mathbb{R}^n \to \mathbb{R}$ is a $L$-Lipschitz function. There exists an algorithm which, given i.i.d. access to labeled samples $(\boldsymbol{x}, f(\boldsymbol{x}))$ where $\boldsymbol{x} \sim N(0, I_n)$, draws $n^{O(L^2/\varepsilon^2)}$ samples, runs in t Additionally, the function $g$ is convex and $L$-Lipschitz.

Figures (2)

  • Figure 1: One-dimensional illustration of the empirical convex envelope (red, dashed) of (a) a convex function and (b) a far-from-convex function, with respect to the sampled points shown in cyan. (See \ref{['def:empirical-convex-envelope']} for a precise definition.)
  • Figure 2: The quadratic program CECE$(x)$ which takes as input a collection of samples $Y = \{y_1, \dots, y_m\}$ labeled by an $L$-Lipschitz function $f$.

Theorems & Definitions (83)

  • Theorem 0: Agnostic proper learning of Lipschitz convex functions
  • Theorem 0: Tolerant two-sided testing of Lipschitz convex functions
  • Theorem 0: CSQ lower bound; informal, see \ref{['thm:sq-lower-bound']}
  • Theorem 0: One-sided testing of Lipschitz convex functions
  • Definition 4: Hermite basis
  • Lemma 6: HSSV21colt
  • Remark 7
  • Lemma 8: Exercise 11.10(a) of odonnell-book
  • Proposition 9
  • Definition 10: Definition 11.12 and Proposition 11.37 of odonnell-book
  • ...and 73 more