The Computational Complexity of Avoiding Strict Saddle Points in Constrained Optimization

Abstract

While first-order stationary points (FOSPs) are the traditional targets of non-convex optimization, they often correspond to undesirable strict saddle points. To circumvent this, attention has shifted towards second-order stationary points (SOSPs). In unconstrained settings, finding approximate SOSPs is PLS-complete (Kontogiannis et al.), matching the complexity of finding unconstrained FOSPs (Hollender and Zampetakis). However, the complexity of finding SOSPs in constrained settings remained notoriously unclear and was highlighted as an important open question by both aforementioned works. Under one strict definition, even verifying whether a point is an approximate SOSP is NP-hard (Murty and Kabadi). Under another widely adopted, relaxed definition where non-negative curvature is required only along the null space of the active constraints, the problem lies in TFNP, and algorithms with O(poly(1/epsilon)) running times have been proposed (Lu et al.). In this work, we settle the complexity of constrained SOSP by proving that computing an epsilon-approximate SOSP under the tractable definition is PLS-complete. We demonstrate that our result holds even in the 2D unit square [0,1]^2, and remarkably, even when stationary points are isolated at a distance of Omega(1) from the domain's boundary. Our result establishes a fundamental barrier: unless PLS is a subset of PPAD (implying PLS = CLS), no deterministic, iterative algorithm with an efficient, continuous update rule can exist for finding approximate SOSPs. This contrasts with the constrained first-order counterpart, for which Fearnley et al. showed that finding an approximate KKT point is CLS-complete. Finally, our result yields the first problem defined in a compact domain to be shown PLS-complete beyond the canonical Real-LocalOpt (Daskalakis and Papadimitriou)."

Figures (21)

• Figure 1: A diagram of an ITER instance with 5 sequentially arranged nodes, labeled 1 to 5 from left to right. The nodes are represented by distinct squares, and curved black arrows indicate the mapping function $C$. Node 3 is a solution because it maps to the fixed point at node 4 ($C(3) = 4$), which means $C(C(3)) = C(4) = 4$. This is a mapping to a right fixed point. Node 5 is a solution because $C(5) = 1$ is a mapping to the left, and $1 < 5$.
• Figure 2: Our reductions.
• Figure 3: A high-level illustration of our construction: We have embedded an abstract Iter instance within the square box $[0,N]^2$, which is the domain of function $f$. Each arrow shows the direction of the negative gradient flow. In this Iter example, node 3 and 5 are solutions, and it holds that $C(1) = 2$, $C(2) = 5$, $C(3) = 4$, $C(4) = 4$ and $C(5) < 5$. The example can be similarly extended in an arbitrary way for all $2^n$ nodes of the generic Iter problem.
• Figure 6: The color value regimes: The colors (B, M, R, G, O) are ordered according to decreasing value, from right to left. The direction of each arrow indicates the direction which decreases the value of the function in the x-y plane.
• Figure 7: A high-level illustration of our construction: We have embedded an abstract Iter instance within the square box $[0,N]^2$, which is the domain of function $f$. Each arrow shows the direction of the negative gradient flow. In this Iter example, node 3 and 5 are solutions, and it holds that $C(1) = 2$, $C(2) = 5$, $C(3) = 4$, $C(4) = 4$ and $C(5) < 5$. The example can be similarly extended in an arbitrary way for all $2^n$ nodes of the generic Iter problem.

Theorems & Definitions (50)

• Theorem
• Theorem : Abridged; Formally stated in Theorem \ref{['thm:main']}
• Remark 1.1
• Definition 2.1
• Definition 2.2
• Definition 2.3
• Definition 2.4
• Theorem : PLS-completeness
• Theorem : There exists no continuous algorithm for finding SOSPs
• Definition 3.1