Efficient Testable Learning of General Halfspaces with Adversarial Label Noise

TL;DR

This work is the first polynomial time tester-learner for general halfspaces that achieves dimension-independent misclassification error and develops a tester-learner such that if the data passes the tester, then one can trust the output of the robust learner on the data.

Abstract

We study the task of testable learning of general -- not necessarily homogeneous -- halfspaces with adversarial label noise with respect to the Gaussian distribution. In the testable learning framework, the goal is to develop a tester-learner such that if the data passes the tester, then one can trust the output of the robust learner on the data.Our main result is the first polynomial time tester-learner for general halfspaces that achieves dimension-independent misclassification error. At the heart of our approach is a new methodology to reduce testable learning of general halfspaces to testable learning of nearly homogeneous halfspaces that may be of broader interest.

Figures (3)

• Figure 1: When there is no noise, the tail points are at distance at least $t^\ast$ from the origin, since they all lie on the side of the hyperplane not containing the origin. Consequently, the line along their mean $\boldsymbol \mu$ must first intersect the separating hyperplane of the halfspace (crossing $\mathbf{w}$) and then cross the mean vector $\boldsymbol \mu$ of the tail points, regardless of the underlying marginal distribution. If we re-center the distribution at $\mathbf{w}$, the halfspace will then become exactly homogeneous.
• Figure 2: The figure illustrates a good localization center $\mathbf{w}$ with respect to the halfspace $h$. $\mathbf{w}$ is $\alpha$-far from the halfspace, and $\Phi(\left\| \mathbf{w} \right\|_{2})$ is still non-trivial (bounded from below by the mass on one side of the halfspace).
• Figure 3: Bound $\left\| \mathbf{v} \right\|_{2} - \left\| \boldsymbol{\mu} \right\|_{2}$ via $\mathbf{v}^\ast \cdot (\mathbf{v} - \boldsymbol{\mu})$.

Theorems & Definitions (31)

• Definition 1.1: Testable Learning with Adversarial Label Noise RV22a
• Theorem 1.2: Testable Learning General Halfspaces under Gaussian Marginals
• Definition 2.1: Good Localization Center
• Definition 2.2
• Lemma 2.3: Localization With A Good Center
• Proposition 2.4: Testable Learning of Nearly Homogeneous Halfspaces
• Lemma 2.5: Transformation Error
• Lemma 2.6: Wedge Bound for General Halfspaces
• Proposition 2.7
• Definition 2.8: Tail point