New complementarity formulations for root-finding and optimization of piecewise-affine functions in abs-normal form

TL;DR

The paper tackles solving root-finding and optimization for continuous piecewise-affine functions expressed in abs-normal form by deriving new complementarity-based reformulations. It shows that root-finding can be cast as a mixed linear complementarity problem (MLCP) and, under mild regularity, reduced to a linear complementarity problem (LCP); optimization is formulated as a linear program with complementarity constraints (LPCC) or MILP via big-M, with existence checks via horizon functions. A Julia prototype demonstrates automatic construction of the requisite auxiliary ANF quantities and solves the reformulated problems with PATH and BARON solvers, highlighting advantages over prior Griewank-style approaches, especially in higher dimensions. These contributions broaden the practical utility of abs-normal forms for nonsmooth analysis and optimization, suggesting paths for integration with AD-based ANF generation and potential applications to neural networks and other discrete-continuous systems.

Abstract

Nonsmooth functions have been used to model discrete-continuous phenomena such as contact mechanics, and are also prevalent in neural network formulations via activation functions such as ReLU. At previous AD conferences, Griewank et al. showed that nonsmooth functions may be approximated well by piecewise-affine functions constructed using an AD-like procedure. Moreover, such a piecewise-affine function may always be represented in an "abs-normal form", encoding it as a collection of four matrices and two vectors. We present new general complementarity formulations for root-finding and optimization of piecewise-affine functions in abs-normal form, with significantly fewer restrictions than previous approaches. In particular, piecewise-affine root-finding may always be represented as a mixed-linear complementarity problem (MLCP), which may often be simplified to a linear complementarity problem (LCP). We also present approaches for verifying existence of solutions to these problems. A proof-of-concept implementation in Julia is discussed and applied to several numerical examples, using the PATH solver to solve complementarity problems.

Figures (4)

• Figure 1: The horizon function $f^\infty$ (dashed) for the piecewise affine function $f$ given by \ref{['eq:horizon']} (solid).
• Figure 2: CPU time (s) for generating a random $\mathcal{PA}$ function in Example \ref{['ex:ScalableRootFinding']} and determining its root using our new root-finding approaches (including computing all necessary vectors, matrices, and solving the corresponding formulations), averaged over 100 runs. The inset figure zooms in on the lower-left corner.
• Figure 3: CPU time (s) for generating a random $\mathcal{PA}$ function in Example \ref{['ex:CompareRootGriewank']} and determining its root using both our new approaches and Griewank et al.'s approaches (including computing all necessary vectors, matrices, and solving the corresponding formulations), averaged over 100 runs. The inset figure zooms in on the lower-left corner.
• Figure 4: CPU time (s) for computing our auxiliary matrices/vectors in Definition \ref{['def:MandV']} for the function $f$ in Example \ref{['ex:ScalableOptimization']} (yellow), and for minimizing it via our MILP formulation (red) and our LPCC formulation (blue), averaged over 100 runs. The inset figure zooms in on the lower-left corner.

Theorems & Definitions (19)

• Definition 2.1
• Example 1
• Proposition 2.1: from GriewankStreubel
• Definition 3.1
• Example 2
• Theorem 3.1
• Corollary 3.1
• Corollary 3.2
• Theorem 4.1
• Corollary 4.1