An Inexact General Descent Method with Applications in Differential Equation-Constrained Optimization

TL;DR

This work addresses optimization with inexact first-order information in differential equation-constrained settings by introducing an inexact general descent framework (IGDM). It establishes global convergence under two step-size regimes and develops two concrete algorithms, Inexact Gradient Descent (IGD) and Inexact BFGS-like (IBFGS), implemented with an adaptive adjoint gradient error control via a test-and-tighten scheme. Theoretical results show convergence to stationary points under bounded and diminishing step sizes, with monotonic descent guaranteed in the bounded regime when gradient-error is proportional to the gradient norm. Numerical experiments on two DECO-based inverse problems demonstrate substantial runtime reductions from adaptive inexact gradients, with curvature information providing further gains. The approach offers practical, scalable optimization in contexts where discrete adjoint gradients are expensive or inexact, and points to future work on nonlinear residuals, constraints, and line-search/trust-region strategies.

Abstract

In many applications, gradient evaluations are inherently approximate, motivating the development of optimization methods that remain reliable under inexact first-order information. A common strategy in this context is adaptive evaluation, whereby coarse gradients are used in early iterations and refined near a minimizer. This is particularly relevant in differential equation-constrained optimization (DECO), where discrete adjoint gradients depend on iterative solvers. Motivated by DECO applications, we propose an inexact general descent framework and establish its global convergence theory under two step-size regimes. For bounded step sizes, the analysis assumes that the error tolerance in the computed gradient is proportional to its norm, whereas for diminishing step sizes, the tolerance sequence is required to be summable. The framework is implemented through inexact gradient descent and an inexact BFGS-like method, whose performance is demonstrated on a second-order ODE inverse problem and a two-dimensional Laplace inverse problem using discrete adjoint gradients with adaptive accuracy. Across these examples, adaptive inexact gradients consistently reduced optimization time relative to fixed tight tolerances, while incorporating curvature information further improved overall efficiency.

Figures (5)

• Figure 1: Optimized solution curve with reference points.
• Figure 2: Optimized solution surface with reference points.
• Figure 3: Optimization time versus interior points $M$ with percentage-reduction curves. IGD uses $\gamma_{1}=60$, $\gamma_{2}=3$, and IBFGS uses $\gamma_{1}=1e-3$, $\gamma_{2}=1e-3$. All methods use constant step size $t=9.583e-2$ and stopping tolerance $\epsilon=1e-6$. Horizontal lines show the mean reduction for $64 \le M \le 128$.
• Figure 4: Wall-clock optimization time as a function of interior points $M$, together with percentage-reduction curves. IGD uses $\gamma_{1}=60$ and $\gamma_{2}=3$. IBFGS uses $\gamma_{1}=1e-1$ and $\gamma_{2}=1e-1$. All methods use stopping tolerance $\epsilon=1e-6$. Horizontal lines indicate the mean reduction for $64 \le M \le 128$.
• Figure :

Theorems & Definitions (16)

• Lemma 4.1
• proof
• Lemma 4.2
• proof
• Remark 4.1
• Lemma 4.3
• proof
• Remark 4.2
• Theorem 4.3
• proof