Fast Agnostic Learners in the Plane

TL;DR

The paper addresses the time complexity of agnostic learning for planar geometric classes—triangles, convex $k$-gons, and convex sets in a square—introducing two-sample reference-concept methods that yield near-optimal sample complexity with significantly improved running times: triangles in $\tilde{O}(\varepsilon^{-6})$, 4-gons in $\tilde{O}(\varepsilon^{-8})$, 5-gons in $\tilde{O}(\varepsilon^{-10})$, and convex sets in a square in $\tilde{O}(\varepsilon^{-5})$. The approach hinges on constructing a small reference set from a base sample and using a larger sample to select a near-optimal reference, combined with efficient geometric data structures for triangle-range queries and the OptIslands framework to handle islands. The results imply tolerant property testers with matching runtimes and reveal a fundamental gap between sample and time complexity for these natural geometric classes, while leaving open whether this gap is intrinsic or can be closed in higher dimensions or for larger $k$. These contributions advance both theory and practice in low-dimensional geometric learning and have implications for robust, image-oriented learning tasks where geometry plays a central role.

Abstract

We investigate the computational efficiency of agnostic learning for several fundamental geometric concept classes in the plane. While the sample complexity of agnostic learning is well understood, its time complexity has received much less attention. We study the class of triangles and, more generally, the class of convex polygons with $k$ vertices for small $k$, as well as the class of convex sets in a square. We present a proper agnostic learner for the class of triangles that has optimal sample complexity and runs in time $\tilde O({ε^{-6}})$, improving on the algorithm of Dobkin and Gunopulos (COLT `95) that runs in time $\tilde O({ε^{-10}})$. For 4-gons and 5-gons, we improve the running time from $O({ε^{-12}})$, achieved by Fischer and Kwek (eCOLT `96), to $\tilde O({ε^{-8}})$ and $\tilde O({ε^{-10}})$, respectively. We also design a proper agnostic learner for convex sets under the uniform distribution over a square with running time $\tilde O({ε^{-5}})$, improving on the previous $\tilde O(ε^{-8})$ bound at the cost of slightly higher sample complexity. Notably, agnostic learning of convex sets in $[0,1]^2$ under general distributions is impossible because this concept class has infinite VC-dimension. Our agnostic learners use data structures and algorithms from computational geometry and their analysis relies on tools from geometry and probabilistic combinatorics. Because our learners are proper, they yield tolerant property testers with matching running times. Our results raise a fundamental question of whether a gap between the sample and time complexity is inherent for agnostic learning of these and other natural concept classes.

Figures (4)

• Figure 3.1: An illustration to Definition \ref{['def:induced-triangles']}: a set of points and a (light blue) reference triangle it defines.
• Figure 3.2: An illustration to Lemma \ref{['lem:eps_net_triangle']}: a (wavy) triangle $t$ and a (light blue) nearby reference triangle.
• Figure 4.1: An illustration of an island: a set of points is intersected with a (light blue) convex set; the (blue) points in the intersection form an island.
• Figure 4.2: The convex hull of the island is delineated by (red) lines; the (six) vertices of the convex hull have the red border.

Theorems & Definitions (41)

• Definition 1.1: Distance Approximation
• Definition 2.1: Agnostic PAC learning
• Definition 2.2: The class of halfplanes
• Definition 2.3: The class of $k$-gons
• Definition 2.4: The class of convex sets
• Definition 2.5
• Definition 2.6: Asymmetric discrepancy
• Claim 2.7: Asymmetric discrepancy vs. empirical risk
• proof
• Definition 2.8: $\varepsilon$-representative set of examples