Deep FlexQP: Accelerated Nonlinear Programming via Deep Unfolding

TL;DR

This work tackles the challenge of solving NLPs with potentially infeasible linearizations by introducing FlexQP, an always-feasible QP solver built on an exact elastic relaxation and ADMM-style splitting. It then enhances FlexQP with Deep FlexQP, a learning-based extension that jointly learns dimension-agnostic feedback policies for penalty and relaxation parameters via deep unfolding, supported by PAC-Bayes generalization guarantees. The approach is validated across diverse problem classes, including portfolio optimization, SVMs, LASSO, Huber fitting, and nonlinear optimal control, demonstrating substantial speedups and robust performance, including in nonconvex NLP via SQP subproblems. The results show that learning-based parameter control yields notable convergence and runtime benefits, with formal certificates providing a practical safety net for deployment in real-time control and predictive safety applications.

Abstract

We propose an always-feasible quadratic programming (QP) optimizer, FlexQP, which is based on an exact relaxation of the QP constraints. If the original constraints are feasible, then the optimizer finds the optimal solution to the original QP. On the other hand, if the constraints are infeasible, the optimizer identifies a solution that minimizes the constraint violation in a sparse manner. FlexQP scales favorably with respect to the problem dimension, is robust to both feasible and infeasible QPs with minimal assumptions on the problem data, and can be effectively warm-started. We subsequently apply deep unfolding to improve our optimizer through data-driven techniques, leading to an accelerated Deep FlexQP. By learning dimension-agnostic feedback policies for the parameters from a small number of training examples, Deep FlexQP generalizes to problems with larger dimensions and can optimize for many more iterations than it was initially trained for. Our approach outperforms two recently proposed state-of-the-art accelerated QP approaches on a suite of benchmark systems including portfolio optimization, classification, and regression problems. We provide guarantees on the expected performance of our deep QP optimizer through probably approximately correct (PAC) Bayes generalization bounds. These certificates are used to design an accelerated sequential quadratic programming solver that solves nonlinear optimal control and predictive safety filter problems faster than traditional approaches. Overall, our approach is very robust and greatly outperforms existing non-learning and learning-based optimizers in terms of both runtime and convergence to the optimal solution across multiple classes of NLPs.

Figures (20)

• Figure 1: SQP with Deep FlexQP can solve highly-constrained nonlinear optimizations over 15x faster than SQP with OSQP (averaged over 100 problems).
• Figure 2: One layer of our proposed Deep FlexQP architecture. We learn dimension-agnostic feedback policies for the parameters while the propagation from one layer to the next is defined by the ADMM updates \ref{['eq:FlexQP_admm_updates']}.
• Figure 3: Log-scaled loss better captures small errors when the solution is close to the optimal.
• Figure 4: Optimizer comparison on 1000 test LASSO problems. Training using our log-normalized loss \ref{['eq:new_gen_bound_loss']} results in a substantially more informative performance guarantee.
• Figure 5: Performance comparison of learned deep optimizers and their non-learned counterparts on small- to medium-scale QPs. Our improved version of Deep OSQP outperforms the baselines, while Deep FlexQP consistently surpasses the rest of the methods in terms of convergence to the optimal QP solution.

Theorems & Definitions (4)

• Theorem 3.1
• Theorem 3.2
• Theorem 3.3
• Lemma 1.1