Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Random-Subspace Sequential Quadratic Programming for Constrained Zeroth-Order Optimization

Runyu Zhang, Gioele Zardini

Abstract

We study nonlinear constrained optimization problems in which only function evaluations of the objective and constraints are available. Existing zeroth-order methods rely on noisy gradient and Jacobian surrogates in high dimensions, making it difficult to simultaneously achieve computational efficiency and accurate constraint satisfaction. We propose a zeroth-order random-subspace sequential quadratic programming method (ZO-RS-SQP) that combines two-point directional estimation with low-dimensional SQP updates. At each iteration, the method samples a random low-dimensional subspace, estimates the projected objective gradient and constraint Jacobians using two-point evaluations, and solves a reduced quadratic program to compute the step. As a result, the per-iteration evaluation cost scales with the subspace dimension rather than the ambient dimension, while retaining the structured linearized-constraint treatment of SQP. We also consider an Armijo line-search variant that improves robustness in practice. Under standard smoothness and regularity assumptions, we establish convergence to first-order KKT points with high probability. Numerical experiments illustrate the effectiveness of the proposed approach on nonlinear constrained problems.

Random-Subspace Sequential Quadratic Programming for Constrained Zeroth-Order Optimization

Abstract

We study nonlinear constrained optimization problems in which only function evaluations of the objective and constraints are available. Existing zeroth-order methods rely on noisy gradient and Jacobian surrogates in high dimensions, making it difficult to simultaneously achieve computational efficiency and accurate constraint satisfaction. We propose a zeroth-order random-subspace sequential quadratic programming method (ZO-RS-SQP) that combines two-point directional estimation with low-dimensional SQP updates. At each iteration, the method samples a random low-dimensional subspace, estimates the projected objective gradient and constraint Jacobians using two-point evaluations, and solves a reduced quadratic program to compute the step. As a result, the per-iteration evaluation cost scales with the subspace dimension rather than the ambient dimension, while retaining the structured linearized-constraint treatment of SQP. We also consider an Armijo line-search variant that improves robustness in practice. Under standard smoothness and regularity assumptions, we establish convergence to first-order KKT points with high probability. Numerical experiments illustrate the effectiveness of the proposed approach on nonlinear constrained problems.

Paper Structure

This paper contains 15 sections, 7 theorems, 75 equations, 3 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Sequential Quadratic Programming
  4. Subspace and Random-Subspace SQP
  5. Zeroth-Order Optimization and Two-Point Estimation
  6. Zeroth-Order Random-Subspace SQP (ZO-RS-SQP)
  7. Merit Function and Armijo Line Search
  8. Convergence Analysis
  9. Acceptance Rate
  10. Descent of the Merit Function
  11. Convergence Rate
  12. Numerical Experiments
  13. Nonlinear Programming
  14. Transient Dynamics Optimization Under Angle Separation Constraints
  15. Conclusion and Future Work

Key Result

Theorem 1

Suppose Assumptions ass:smoothness--ass:orig_regularity hold, and let $d \ge m_e+1$. At iteration $t$, let $U_t$ be a i.i.d. sample generated from Algorithm alg:rssqp, and define the event Then there exists constant $\Lambda^*, M^*$ and $p_{\mathrm{acc}} >0$ (which only depends on the parameters in Assumption ass:smoothness--ass:orig_regularity) such that for any $\Lambda, M \ge \Lambda^*, M^*$ w

Figures (3)

  • Figure E1: Learning curves over iterations: (left) objective value, (middle) constraint violation, and (right) KKT gap.
  • Figure E2: Learning curves over iterations: (left) objective value, (middle) constraint violation, and (right) KKT gap.
  • Figure E3: Transient phase (offset) and frequency dynamics under a fault disturbance. Top: solution obtained ZO-RS-S-LS. Bottom: baseline QP solution ignoring angle separation constraints.

Theorems & Definitions (16)

  • Definition 1: Mangasarian--Fromovitz constraint qualification
  • Theorem 1: Positive acceptance probability
  • proof
  • Theorem 2: Decrease of Merit Function per-step
  • proof
  • Theorem 3: One-step residual bounds
  • proof
  • Theorem 4: High-probability $O(T^{-1/2})$ convergence
  • proof
  • Remark 1: On the subspace dimension $d$
  • ...and 6 more