A novel numerical method tailored for unconstrained optimization problems

TL;DR

The paper addresses unconstrained optimization for both smooth and nonsmooth objectives common in scientific computing. It introduces a variationally refined quadratic surrogate with $2n$ constraints, plus a simplified $2$-constraint variant, to efficiently approximate $f$ locally through updated gradients $\boldsymbol{g}^{(k)}$ and Hessians $\boldsymbol{G}^{(k)}$. A convex objective $\Phi$ with Lagrange-multiplier-based updates yields explicit expressions for the increments $\triangle\boldsymbol{g}^{(k)}$ and $\triangle\boldsymbol{G}}^{(k)}$, and a simplified form provides direct updates that can reduce computational cost; convergence and derivative-approximation bounds are established under Lipschitz continuity of the Hessian. The proposed algorithm is validated through numerical experiments on smooth, derivative-blasting, and nonsmooth problems, showing superior speed and accuracy compared with traditional Trust-Region and derivative-free methods, and demonstrating robustness to challenging objective landscapes. These results indicate a practical, easy-to-implement approach for fast unconstrained optimization in complex engineering and scientific applications, with potential extensions to large-scale and sparse nonsmooth problems.

Abstract

Unconstrained optimization problems become more common in scientific computing and engineering applications with the rapid development of artificial intelligence, and numerical methods for solving them more quickly and efficiently have been getting more attention and research. Moreover, an efficient method to minimize all kinds of objective functions is urgently needed, especially the nonsmooth objective function. Therefore, in the current paper, we focus on proposing a novel numerical method tailored for unconstrained optimization problems whether the objective function is smooth or not. To be specific, based on the variational procedure to refine the gradient and Hessian matrix approximations, an efficient quadratic model with $2n$ constrained conditions is established. Moreover, to improve the computational efficiency, a simplified model with 2 constrained conditions is also proposed, where the gradient and Hessian matrix can be explicitly updated, and the corresponding boundedness of the remaining $2n-2$ constrained conditions is derived. On the other hand, the novel numerical method is summarized, and approximation results on derivative information are also analyzed and shown. Numerical experiments involving smooth, derivative blasting, and non-smooth problems are tested, demonstrating its feasibility and efficiency. Compared with existing methods, our proposed method can efficiently solve smooth and non-smooth unconstrained optimization problems for the first time, and it is very easy to program the code, indicating that our proposed method not also has great application prospects, but is also very meaningful to explore practical complex engineering and scientific problems.

Figures (4)

• Figure 3.1: A configuration of our proposed algorithm for computing \ref{['eq2024121301']}.
• Figure 4.1: Some problems with derivative blasting.
• Figure 4.2: $\phi$ vs $t$ in \ref{['eq2025022102']}.
• Figure 4.3: $u(r)$ vs $r$ for \ref{['2024072501']}.

Theorems & Definitions (2)

• proof
• proof