Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Finite-Difference Trust-Region Method for Convexly Constrained Smooth Optimization

Dânâ Davar, Geovani Nunes Grapiglia

TL;DR

This work introduces TRFD-S, a derivative-free trust-region method for smooth optimization with convex constraints that relies on finite-difference gradient estimates and a quadratic model to achieve efficient, theoretically grounded convergence guarantees. It provides explicit worst-case function-evaluation bounds across nonconvex, convex, and Polyak-Lojasiewicz settings, highlighting a trade-off controlled by the Lipschitz constant $L$ and the gradient-estimator scale $\sigma$. The method is extended to unrelaxable bound constraints and validated through extensive numerical experiments showing competitive performance against state-of-the-art derivative-free solvers on unconstrained and constrained problems, including an ODE-model calibration example. Overall, TRFD-S offers a practical balance between per-iteration cost and convergence guarantees in black-box smooth optimization under convex constraints, with broad applicability to parameter estimation and model fitting tasks.

Abstract

We propose a derivative-free trust-region method based on finite-difference gradient approximations for smooth optimization problems with convex constraints. The proposed method does not require computing an approximate stationarity measure. For nonconvex problems, we establish a worst-case complexity bound of $\mathcal{O}\!\left(n\left(\tfrac{L}σε\right)^{-2}\right)$ function evaluations for the method to reach an $\left(\tfrac{L}σε\right)$-approximate stationary point, where $n$ is the number of variables, $L$ is the Lipschitz constant of the gradient, and $σ$ is a user-defined estimate of $L$. If the objective function is convex, the complexity to reduce the functional residual below $(L/σ)ε$ is shown to be of $\mathcal{O}\!\left(n\left(\tfrac{L}σε\right)^{-1}\right)$ function evaluations, while for Polyak-Lojasiewicz functions on unconstrained domains, the bound further improves to $\mathcal{O}\left(n\log\left(\left(\frac{L}σε\right)^{-1}\right)\right)$. Numerical experiments on benchmark problems and a model-fitting application demonstrate the method's efficiency relative to state-of-the-art derivative-free solvers for both unconstrained and bound-constrained problems.

A Finite-Difference Trust-Region Method for Convexly Constrained Smooth Optimization

TL;DR

This work introduces TRFD-S, a derivative-free trust-region method for smooth optimization with convex constraints that relies on finite-difference gradient estimates and a quadratic model to achieve efficient, theoretically grounded convergence guarantees. It provides explicit worst-case function-evaluation bounds across nonconvex, convex, and Polyak-Lojasiewicz settings, highlighting a trade-off controlled by the Lipschitz constant and the gradient-estimator scale . The method is extended to unrelaxable bound constraints and validated through extensive numerical experiments showing competitive performance against state-of-the-art derivative-free solvers on unconstrained and constrained problems, including an ODE-model calibration example. Overall, TRFD-S offers a practical balance between per-iteration cost and convergence guarantees in black-box smooth optimization under convex constraints, with broad applicability to parameter estimation and model fitting tasks.

Abstract

We propose a derivative-free trust-region method based on finite-difference gradient approximations for smooth optimization problems with convex constraints. The proposed method does not require computing an approximate stationarity measure. For nonconvex problems, we establish a worst-case complexity bound of function evaluations for the method to reach an -approximate stationary point, where is the number of variables, is the Lipschitz constant of the gradient, and is a user-defined estimate of . If the objective function is convex, the complexity to reduce the functional residual below is shown to be of function evaluations, while for Polyak-Lojasiewicz functions on unconstrained domains, the bound further improves to . Numerical experiments on benchmark problems and a model-fitting application demonstrate the method's efficiency relative to state-of-the-art derivative-free solvers for both unconstrained and bound-constrained problems.
Paper Structure (15 sections, 24 theorems, 123 equations, 4 figures, 3 tables)

This paper contains 15 sections, 24 theorems, 123 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Problem and Contributions
  3. Contents
  4. Assumptions and Auxiliary Results
  5. Trust-Region Method for Relaxable Convex Constraints
  6. Worst-Case Complexity Bound for Nonconvex Problems
  7. Worst-Case Complexity Bound for Convex Problems
  8. Worst-Case Complexity Bound for P-L functions
  9. Trust-Region Method for Unrelaxable Bound Constraints
  10. Numerical Experiments
  11. Benchmark Problems
  12. Unconstrained Problems
  13. Unrelaxable Bound Constraints Problems
  14. Calibration of an ODE Model
  15. Conclusion

Key Result

Lemma 2.2

(Lemma 2.6 in grapiglia). Suppose that A1 and A2 hold, and let $\psi_r$ be defined by psi. Then, (a) $\psi_{r}(x) \geq 0, \quad \forall x \in \Omega$; (b) $\psi_{r}(x^{*})=0$ if, and only if, $x^*$ is a stationary point of $f$ in $\Omega$.

Figures (4)

  • Figure 1: Illustration of Step 1 for unrelaxable bound constraints
  • Figure 2: Data profiles for smooth unconstrained problems
  • Figure 3: Data profiles for smooth unrelaxable bound constraints problems
  • Figure 4: Model Fitting on a Synthetic Predator–Prey Dataset

Theorems & Definitions (44)

  • Definition 2.1
  • Lemma 2.2
  • Lemma 2.3
  • Remark 3.1
  • Lemma 3.2
  • proof
  • Lemma 3.3
  • proof
  • Lemma 3.4
  • Lemma 3.5
  • ...and 34 more