Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sandwiching Polynomials for Geometric Concepts with Low Intrinsic Dimension

Adam R. Klivans, Konstantinos Stavropoulos, Arsen Vasilyan

TL;DR

A new method for constructing low-degree sandwiching polynomials that yield greatly improved degree bounds for several fundamental function classes and marginal distributions is given and directly uses the smoothness of the target function's boundary to construct sandwiching Lipschitz functions.

Abstract

Recent work has shown the surprising power of low-degree sandwiching polynomial approximators in the context of challenging learning settings such as learning with distribution shift, testable learning, and learning with contamination. A pair of sandwiching polynomials approximate a target function in expectation while also providing pointwise upper and lower bounds on the function's values. In this paper, we give a new method for constructing low-degree sandwiching polynomials that yield greatly improved degree bounds for several fundamental function classes and marginal distributions. In particular, we obtain degree $\mathrm{poly}(k)$ sandwiching polynomials for functions of $k$ halfspaces under the Gaussian distribution, improving exponentially over the prior $2^{O(k)}$ bound. More broadly, our approach applies to function classes that are low-dimensional and have smooth boundary. In contrast to prior work, our proof is relatively simple and directly uses the smoothness of the target function's boundary to construct sandwiching Lipschitz functions, which are amenable to results from high-dimensional approximation theory. For low-dimensional polynomial threshold functions (PTFs) with respect to Gaussians, we obtain doubly exponential improvements without applying the FT-mollification method of Kane used in the best previous result.

Sandwiching Polynomials for Geometric Concepts with Low Intrinsic Dimension

TL;DR

A new method for constructing low-degree sandwiching polynomials that yield greatly improved degree bounds for several fundamental function classes and marginal distributions is given and directly uses the smoothness of the target function's boundary to construct sandwiching Lipschitz functions.

Abstract

Recent work has shown the surprising power of low-degree sandwiching polynomial approximators in the context of challenging learning settings such as learning with distribution shift, testable learning, and learning with contamination. A pair of sandwiching polynomials approximate a target function in expectation while also providing pointwise upper and lower bounds on the function's values. In this paper, we give a new method for constructing low-degree sandwiching polynomials that yield greatly improved degree bounds for several fundamental function classes and marginal distributions. In particular, we obtain degree sandwiching polynomials for functions of halfspaces under the Gaussian distribution, improving exponentially over the prior bound. More broadly, our approach applies to function classes that are low-dimensional and have smooth boundary. In contrast to prior work, our proof is relatively simple and directly uses the smoothness of the target function's boundary to construct sandwiching Lipschitz functions, which are amenable to results from high-dimensional approximation theory. For low-dimensional polynomial threshold functions (PTFs) with respect to Gaussians, we obtain doubly exponential improvements without applying the FT-mollification method of Kane used in the best previous result.
Paper Structure (35 sections, 24 theorems, 61 equations, 1 figure, 1 table)

This paper contains 35 sections, 24 theorems, 61 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Our Results
  3. Our Techniques
  4. Sandwiching Geometric Concepts by Lipschitz Functions
  5. Sandwiching Polynomials for Lipschitz Functions
  6. Comparison to gopalan2010fooling
  7. Applications
  8. Tolerant Testable Learning
  9. Learning with Distribution Shift
  10. Learning with Heavy Contamination
  11. Related Work
  12. Pseudorandomness
  13. Learning via Polynomial Approximation
  14. Learning via Sandwiching
  15. Learning via Boundary Smoothness
  16. ...and 20 more sections

Key Result

Theorem 1.1

The $(\epsilon,s)$-sandwiching degree of concepts with intrinsic dimesion $k$ and $\sigma$-smooth boundary with respect to a $\gamma$-strictly subexponential distribution $\mathcal{D}$ is:

Figures (1)

  • Figure 1: Our upper sandwiching polynomial for $f_{\mathrm{up}}$ is of the form $p_{\mathrm{up}} = p_1(\mathbf{x})+p_2(\mathbf{x})+\epsilon$. In region $\mathcal{X}_1$, $p_1$ is a pointwise $\epsilon$-approximator for $f_{\mathrm{up}}$ and $0\le p_2(\mathbf{x})\le \epsilon$, so $f_{\mathrm{up}}(\mathbf{x})\le p_{\mathrm{up}}(\mathbf{x})\le f_{\mathrm{up}}(\mathbf{x})+ 3\epsilon$. In region $\mathcal{X}_2$, $p_1$ is still an pointwise $\epsilon$-approximator for $f_{\mathrm{up}}$ and $p_2(\mathbf{x}) > 0$, so $f_{\mathrm{up}}(\mathbf{x}) \le p_1(\mathbf{x})+\epsilon \le p_{\mathrm{up}}(\mathbf{x})$. In region $\mathcal{X}_3$ we have $p_2(\mathbf{x}) \ge 1+|p_1(\mathbf{x})| \ge f_{\mathrm{up}}(\mathbf{x})+|p_1(\mathbf{x})|$, so $p_{\mathrm{up}}(\mathbf{x}) \ge f_{\mathrm{up}}(\mathbf{x})$.

Theorems & Definitions (53)

  • Definition 1: Sandwiching Degree
  • Theorem 1.1: Main Theorem
  • Definition 2: Dilation and Erosion
  • Definition 3: Smooth Boundary chandrasekaran2024efficient
  • Remark 1
  • Definition 4: Strictly Subexponential Distributions
  • Theorem 3.1: Main Theorem, restated
  • Lemma 1
  • proof
  • Lemma 2: chandrasekaran2025learningNewman1964ben2018classical
  • ...and 43 more