Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Functions of Halfspaces

Josh Alman, Shyamal Patel, Rocco A. Servedio

TL;DR

This is the first algorithm that can PAC learn even intersections of two halfspaces in time, running in time 2^{\sqrt{n} \cdot (\log n)^{O(k)}.$

Abstract

We give an algorithm that learns arbitrary Boolean functions of $k$ arbitrary halfspaces over $\mathbb{R}^n$, in the challenging distribution-free Probably Approximately Correct (PAC) learning model, running in time $2^{\sqrt{n} \cdot (\log n)^{O(k)}}$. This is the first algorithm that can PAC learn even intersections of two halfspaces in time $2^{o(n)}.$

Learning Functions of Halfspaces

TL;DR

This is the first algorithm that can PAC learn even intersections of two halfspaces in time, running in time 2^{\sqrt{n} \cdot (\log n)^{O(k)}.$

Abstract

We give an algorithm that learns arbitrary Boolean functions of arbitrary halfspaces over , in the challenging distribution-free Probably Approximately Correct (PAC) learning model, running in time . This is the first algorithm that can PAC learn even intersections of two halfspaces in time
Paper Structure (28 sections, 19 theorems, 80 equations, 2 figures, 2 algorithms)

This paper contains 28 sections, 19 theorems, 80 equations, 2 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Our result
  3. Our techniques
  4. Warm-up: Learning an intersection of two halfspaces.
  5. Learning functions of $k$ halfspaces.
  6. Preliminaries
  7. Basic tools from probability
  8. Origin-Centered Halfspaces
  9. Margin and Radial Isotropic Position
  10. Warm-Up: Learning an Intersection of Two Halfspaces
  11. Achieving Nontrivial Accuracy on the Input Data set $S$
  12. The Advantage Lemma
  13. Filtering
  14. Achieving Non-Trivial Accuracy on the Sample $S$
  15. Learning Functions of Halfspaces
  16. ...and 13 more sections

Key Result

Theorem 1

There is an algorithm that runs in time $\mathrm{poly}(2^{\sqrt{n} \cdot (\log n)^{O(k)}},$$1/\varepsilon,$$\log(1/\delta))$ and learns ${\cal C}_k$ in the distribution-free Probably Approximately Correct (PAC) learning model, using random examples only.

Figures (2)

  • Figure 1: Description and Pointers for Notation For the Algorithm and Its Analysis.
  • Figure 2: Example depictions of the leaderboard

Theorems & Definitions (51)

  • Theorem 1
  • Lemma 2: Gaussian tail bound
  • Lemma 3: Reverse Markov
  • Definition 4: Margin
  • Definition 5: $(1 + \varepsilon)$-Radial Isotropic Position with respect to $V$
  • Definition 6
  • Lemma 7
  • proof
  • Theorem 8: Algorithmic Forster Transform diakonikolas2023strongly
  • Remark 9: The Forster transform is compatible with origin-centered halfspaces
  • ...and 41 more