Testing Approximate Stationarity Concepts for Piecewise Affine Functions
Lai Tian, Anthony Man-Cho So
TL;DR
This work investigates the computational problem of detecting approximate stationarity for continuous piecewise affine functions, focusing on Clarke subdifferentials and NAS concepts. It establishes strong negative results showing NP- and co-NP-hardness for exact and approximate stationarity testing under common PA representations, and introduces a practical relaxation (SRR) together with a tight decision criterion based on polyhedral compatibility for the exact subdifferential sum rule. To overcome intractability, the authors develop a robust NAS-testing framework with a projection-based rounding scheme (the Butterfly Net), delivering oracle-polynomial-time termination guarantees under a geometric separation condition. They also introduce a necessary-and-sufficient condition for Clarke sum-rule validity in the PA setting and connect transversality with zonotopes to sharpen sufficient conditions and enable efficient verification in many practical PA models. The paper culminates with applications to ρ-margin SVM, piecewise affine regression, and shallow/nonsmooth neural networks, illustrating the broad impact of the results on structured PA problems in optimization and machine learning.
Abstract
We study the basic computational problem of detecting approximate stationary points for continuous piecewise affine (PA) functions. Our contributions span multiple aspects, including complexity, regularity, and algorithms. Specifically, we show that testing first-order approximate stationarity concepts, as defined by commonly used generalized subdifferentials, is computationally intractable unless P=NP. To facilitate computability, we consider a polynomial-time solvable relaxation by abusing the convex subdifferential sum rule and establish a tight characterization of its exactness. Furthermore, addressing an open issue motivated by the need to terminate the subgradient method in finite time, we introduce the first oracle-polynomial-time algorithm to detect so-called near-approximate stationary points for PA functions. A notable byproduct of our development in regularity is the first necessary and sufficient condition for the validity of an equality-type (Clarke) subdifferential sum rule. Our techniques revolve around two new geometric notions for convex polytopes and may be of independent interest in nonsmooth analysis. Moreover, some corollaries of our work on complexity and algorithms for stationarity testing address open questions in the literature. To demonstrate the versatility of our results, we complement our findings with applications to a series of structured piecewise smooth functions, including $ρ$-margin-loss SVM, piecewise affine regression, and nonsmooth neural networks.