Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generalized Adjoint Method

Andrew Zheng, Adam R. Stinchcombe

TL;DR

The paper extends the adjoint method to optimization problems with complex, non-holomorphic cost and constraint functions by leveraging CR-calculus. It derives a general gradient formula under equality constraints, shows how to solve for the necessary sensitivities via constraint differentials, and presents a Lagrangian formulation that yields the same results in a unified framework. Through holomorphic and non-holomorphic examples, including a 1D Helmholtz inverse problem, the method demonstrates accurate gradient computation and practical efficiency gains over finite differences. The work enables robust gradient-based optimization for complex-valued problems in electromagnetism, signal processing, and PDE-constrained settings, where traditional real-variable reformulations are suboptimal. Potential extensions include Hessians, higher-order derivatives, and automatic differentiation strategies tailored to non-holomorphic CR-calculus.

Abstract

The adjoint method is an efficient way to numerically compute gradients in optimization problems with constraints, but is only formulated to differentiable cost and constraint functions on real variables. With the introduction of complex variables, which occur often in many inverse problems in electromagnetism and signal processing problems, both the cost and constraint can become non-holomorphic and hence non-differentiable in the standard definitions. Using the notion of CR-calculus, a generalized adjoint method is introduced that can compute the direction of steepest ascent for the cost function while enforcing the constraint even if both are non-holomorphic.

Generalized Adjoint Method

TL;DR

The paper extends the adjoint method to optimization problems with complex, non-holomorphic cost and constraint functions by leveraging CR-calculus. It derives a general gradient formula under equality constraints, shows how to solve for the necessary sensitivities via constraint differentials, and presents a Lagrangian formulation that yields the same results in a unified framework. Through holomorphic and non-holomorphic examples, including a 1D Helmholtz inverse problem, the method demonstrates accurate gradient computation and practical efficiency gains over finite differences. The work enables robust gradient-based optimization for complex-valued problems in electromagnetism, signal processing, and PDE-constrained settings, where traditional real-variable reformulations are suboptimal. Potential extensions include Hessians, higher-order derivatives, and automatic differentiation strategies tailored to non-holomorphic CR-calculus.

Abstract

The adjoint method is an efficient way to numerically compute gradients in optimization problems with constraints, but is only formulated to differentiable cost and constraint functions on real variables. With the introduction of complex variables, which occur often in many inverse problems in electromagnetism and signal processing problems, both the cost and constraint can become non-holomorphic and hence non-differentiable in the standard definitions. Using the notion of CR-calculus, a generalized adjoint method is introduced that can compute the direction of steepest ascent for the cost function while enforcing the constraint even if both are non-holomorphic.
Paper Structure (12 sections, 77 equations, 3 figures, 1 table)

This paper contains 12 sections, 77 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Foundations of complex differentials
  3. The Generalized Adjoint Method
  4. Holomorphic in $x$ but not $p$
  5. When everything but the cost function is holomorphic
  6. Holomorphic linear constraints
  7. A Lagrangian Formulation
  8. Examples
  9. Holomorphic Example
  10. Non-holomorphic Example
  11. Example for 1-dimensional Helmholtz
  12. Conclusion

Figures (3)

  • Figure 1: A surface plot of the cost function $f(z(p)) = |z|^2$ with constraint \ref{['eq: linear holomorphic constraint']}.
  • Figure 2: A zoomed in picture of \ref{['fig: non-holomorphic contour plot']} showing the saddle point at $p \approx 0.4 + 0.3i$.
  • Figure 3: The true solution to the 1D Helmholtz equation with boundary condition with source term $\sin(2\pi x)$ and Neumann boundary condition $u'(0) = 0.5 - 0.5i$ and $u'(1) = -0.25 - 0.25i$ is shown in a). The Neumann boundary conditions are shown with vectors. The solution was obtained using finite difference with 1000 equi-spaced quadrature points. The contour plot of $\log_{10}(f(p))$ is shown in b) along with a vector field of the gradient. All gradient vectors are scaled equally so that the vectors do not overlap each other. The optimization plane is relatively flat close to the global minimum, which results in the vectors being very small in magnitude. Even when the optimizer gets very close to the global minimum, the slope is not that steep.