Verification of Sequential Convex Programming for Parametric Non-convex Optimization

TL;DR

This work addresses the challenge of providing global worst-case guarantees for sequential convex programming methods applied to parametric nonconvex problems. It introduces a verification framework that encodes SCP steps, parameters, and initialization into a single optimization problem, enabling exact offline certification of suboptimality, constraint violations, and subproblem feasibility. The framework supports a broad class of SCP algorithms, including trust-region, CCP, prox-linear, and relax-round-polish, and demonstrates its utility across control, signal processing, and operations research with numerous numerical case studies. The results offer deep insights into how initialization, parameter sets, and hyperparameters influence worst-case performance and provide a tool for algorithm design and parameter tuning in real-time applications.

Abstract

We introduce a verification framework to exactly verify the worst-case performance of sequential convex programming (SCP) algorithms for parametric non-convex optimization. The verification problem is formulated as an optimization problem that maximizes a performance metric (e.g., the suboptimality after a given number of iterations) over parameters constrained to be in a parameter set and iterate sequences consistent with the SCP update rules. Our framework is general, extending the notion of SCP to include both conventional variants such as trust-region, convex-concave, and prox-linear methods, and algorithms that combine convex subproblems with rounding steps, as in relaxing and rounding schemes. Unlike existing analyses that may only provide local guarantees under limited conditions, our framework delivers global worst-case guarantees--quantifying how well an SCP algorithm performs across all problem instances in the specified family. Applications in control, signal processing, and operations research demonstrate that our framework provides, for the first time, global worst-case guarantees for SCP algorithms in the parametric setting.

Figures (14)

• Figure 1: Box QP results. Left: Results for $\mathcal{X}_1 = [2,4]^d$. The warm-started initialization is guaranteed to achieve global optimality after $14$ iterations (within tolerance $10^{-7}$), but the cold-started initialization is not guaranteed to reach a globally optimal solution. Right: Results for $\mathcal{X}_2 = [5,8]^d$. The cold-started initialization is guaranteed to exactly achieve global optimality after $5$ iterations (the blue curve is not visible because the suboptimality is below $10^{-15}$), but the warm-started initialization is not guaranteed to reach a globally optimal solution. Takeaway message: It is not obvious a priori whether warm-starting yields better results than cold-starting, or whether the trust-region method reaches global optimality. Our framework provides definitive guarantees that can answer these questions by explicitly accounting for the initialization, parameter set, and the trust-region size.
• Figure 2: Network utility results. A larger trust-region size enables the trust-region method to escape poorer local minima that the variants with a smaller trust-region size converge to—an interesting outcome, since larger steps are often associated with instability rather than improved convergence.
• Figure 3: Power converter control results. Left: Results for distance to optimality inexactness. Right: Results for KKT inexactness. Our framework allows us to quantitatively compare these two common stopping criteria. Provided a specific $\epsilon$ value, the distance to optimality inexactness criterion yields the stronger guarantees of the two.
• Figure 4: Knapsack results. All of the verification curves overlap with each other across all $\tau_0$ and $\kappa$ values. In all cases, the algorithm makes progress for $1$ iteration and then stalls. When $\kappa=2$, the final penalty value becomes very large (which is designed to prioritize feasibility lipp2016variations), yet the algorithm still fails to find a feasible solution. This example shows how our framework can reveal cases in which the SCP algorithm is ineffective. Finally, for all $\kappa=2$ settings, the verified performance matches the sample maximum exactly.
• Figure 5: Phase retrieval results. With the smaller parameter set $\mathcal{X}_2$, we can certify global optimality (up to tolerance $0.001$) after $4$ iterations. The worst-case guarantee for the larger parameter set $\mathcal{X}_1$ is significantly worse. For $K=4$ and $\mathcal{X}_2$, Gurobi fails to reach a $2\%$ optimality gap within the $2$ hour time limit, so we report the best upper bound obtained.

Theorems & Definitions (9)

• Proposition 4: Encoding binary variables
• Proposition 5: Encoding sparsity constraints
• Theorem 7
• Lemma 8: Farkas Lemma
• Theorem 9
• proof
• proof
• proof
• proof