Improved Hardness Results for Learning Intersections of Halfspaces
Stefan Tiegel
TL;DR
This work establishes strong hardness for weakly learning intersections of a small number of halfspaces in high dimensions. It derives statistically and computationally significant lower bounds under standard lattice hardness (GapSVP/SIVP) that rule out polynomial-time learning of ω(log log N) halfspaces, and provides unconditional SQ-hardness results for even constant numbers of halfspaces, requiring either extremely fine accuracy or exponentially many SQ queries. A core technical contribution is a novel reduction framework that links intersections of halfspaces to the parallel pancakes distribution, enabling both SQ lower bounds and a reduction-based hardness pathway via higher-degree polynomial threshold functions. The results substantially narrow the gap between known hardness for many halfspaces and the elusive two-halfspaces case, and they introduce techniques that may inform future unconditional and standard-assumption hardness results in high-dimensional learning. The work suggests that higher-degree PTF constructions can yield stronger lower bounds and motivates seeking hard instances beyond low-degree representations to further limit the feasibility of efficient learning.
Abstract
We show strong (and surprisingly simple) lower bounds for weakly learning intersections of halfspaces in the improper setting. Strikingly little is known about this problem. For instance, it is not even known if there is a polynomial-time algorithm for learning the intersection of only two halfspaces. On the other hand, lower bounds based on well-established assumptions (such as approximating worst-case lattice problems or variants of Feige's 3SAT hypothesis) are only known (or are implied by existing results) for the intersection of super-logarithmically many halfspaces [KS09,KS06,DSS16]. With intersections of fewer halfspaces being only ruled out under less standard assumptions [DV21] (such as the existence of local pseudo-random generators with large stretch). We significantly narrow this gap by showing that even learning $ω(\log \log N)$ halfspaces in dimension $N$ takes super-polynomial time under standard assumptions on worst-case lattice problems (namely that SVP and SIVP are hard to approximate within polynomial factors). Further, we give unconditional hardness results in the statistical query framework. Specifically, we show that for any $k$ (even constant), learning $k$ halfspaces in dimension $N$ requires accuracy $N^{-Ω(k)}$, or exponentially many queries -- in particular ruling out SQ algorithms with polynomial accuracy for $ω(1)$ halfspaces. To the best of our knowledge this is the first unconditional hardness result for learning a super-constant number of halfspaces. Our lower bounds are obtained in a unified way via a novel connection we make between intersections of halfspaces and the so-called parallel pancakes distribution [DKS17,BLPR19,BRST21] that has been at the heart of many lower bound constructions in (robust) high-dimensional statistics in the past few years.