A novel switched systems approach to nonconvex optimisation

TL;DR

This work introduces a continuous-time switched-dynamics framework for nonconvex constrained optimisation that converges to a KKT point while estimating only the primal variable, thereby reducing problem dimensionality. The switching law selects active inequality constraints to enforce feasibility, achieving positive invariance of the feasible set and stability via a common Lyapunov function. Theoretical guarantees are provided under a technical assumption (Assumption 1) and are demonstrated through quadratic programming, Rosenbrock minimisation, and energy-efficient building control, with additional insights on convergence and local optimality. Practically, the approach offers a path to real-time, primal-only online optimisation in nonconvex settings, with clear avenues for extension to distributed and rate-of-convergence analyses.

Abstract

We develop a novel switching dynamics that converges to the Karush-Kuhn-Tucker (KKT) point of a nonlinear optimisation problem. This new approach is particularly notable for its lower dimensionality compared to conventional primal-dual dynamics, as it focuses exclusively on estimating the primal variable. Our method is successfully illustrated on general quadratic optimisation problems, the minimisation of the classical Rosenbrock function, and a nonconvex optimisation problem stemming from the control of energy-efficient buildings.

Figures (5)

• Figure 1: Height map of Rosenbrock's function with an inequality constraint. At the point $z_a$, the steepest descent direction intersects the constrained space, whereas at the point $z_b$, the steepest descent gradient does not intersect the constrained space.
• Figure 2: The trajectory of the optimisation dynamics seeking the minimiser of a QP problem subject to linear inequality constraints.
• Figure 3: The trajectory of the optimisation dynamics seeking the minimiser of Rosenbrock's function.
• Figure 4: The trajectory of the optimisation dynamics seeking a solution to the optimisation problem related to energy-efficient buildings. The first plot shows the evolution of the optimisation dynamics, whereas the second plot shows the convergence of the equality constraints to the origin. The solid line indicate the results for tuning gains $\kappa_1 = 1$, $\kappa_2 = 1$, $\delta T = 0.1$, whereas the dashed lines indicate the results for tuning gains $\kappa_1 = 3$, $\kappa_2 = 3$, $\delta T = 0.1$.
• Figure 5: The trajectories generated by the proposed optimisation dynamics and three different discrete-time optimisation algorithms. For each of the SQP, Interior point and Active set methods, the solutions are defined discretely and the steps of the solver are indicated by the $\texttt{o}$ symbols. The optimsation dynamics defines a full trajectory, which has been approximated by Matlab's ODE23s solver and plotted as a solid line in the above figure.

Theorems & Definitions (28)

• Remark 1
• Remark 2
• Theorem 3: Theorem 12.1 nocedalNumericalOptimization1999
• Theorem 4: nocedalNumericalOptimization1999
• Definition 5: Common Lyapunov function bacciottiInvariancePrincipleNonlinear2005
• Definition 6: Weakly invariant set mancilla-aguilarInvariancePrinciplesSwitched2011
• Remark 8
• Remark 9
• Remark 10
• Lemma 11