TRFD: A derivative-free trust-region method based on finite differences for composite nonsmooth optimization

TL;DR

The paper introduces TRFD, a derivative-free trust-region method for minimizing composite functions $f(x)=h(F(x))$ where $F$ is accessible only via a zeroth-order oracle. TRFD builds a forward finite-difference Jacobian approximation $A_k$ and updates the finite-difference stepsize and trust-region radius to bound the modeling error, yielding provable evaluation-complexity bounds for nonconvex and convex settings. In particular, for L1 and Minimax structures, TRFD achieves $ ext{O}(n ε^{-2})$ evaluations to reach an $oldsymbol{ε}$-approximate stationary point, while in monotone convex cases the bound improves to $ ext{O}(n ε^{-1})$, with Minimax cases attaining $ ext{O}(n ε^{-1})$. Numerical experiments on L1 and Minimax problems show TRFD often surpasses or matches existing derivative-free solvers, highlighting its practical effectiveness and the value of finite-difference Jacobian approximations in composite nonsmooth optimization.

Abstract

In this work we present TRFD, a derivative-free trust-region method based on finite differences for minimizing composite functions of the form $f(x)=h(F(x))$, where $F$ is a black-box function assumed to have a Lipschitz continuous Jacobian, and $h$ is a known convex Lipschitz function, possibly nonsmooth. The method approximates the Jacobian of $F$ via forward finite differences. We establish an upper bound for the number of evaluations of $F$ that TRFD requires to find an $ε$-approximate stationary point. For L1 and Minimax problems, we show that our complexity bound reduces to $\mathcal{O}(nε^{-2})$ for specific instances of TRFD, where $n$ is the number of variables of the problem. Assuming that $h$ is monotone and that the components of $F$ are convex, we also establish a worst-case complexity bound, which reduces to $\mathcal{O}(nε^{-1})$ for Minimax problems. Numerical results are provided to illustrate the relative efficiency of TRFD in comparison with existing derivative-free solvers for composite nonsmooth optimization.

Figures (3)

• Figure 1: Data profiles of TRFD-L1, MS-P and DFL1S on L1 problems
• Figure 2: Data profiles of TRFD-M, MS-P and DFMS on Minimax problems
• Figure 3: Data profiles of TRFD-M2, MS-P and TRFD-M on Minimax problems

Theorems & Definitions (46)

• Remark 2.1
• Remark 2.2
• Lemma 2.3
• proof
• Definition 2.4
• Lemma 2.5
• proof
• Remark 2.6
• Lemma 2.7
• proof