Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The complex-step Newton method and its convergence

Dimitrios Mitsotakis

TL;DR

This work analyzes the complex-step Newton method for solving $f(x)=0$ and $oldsymbol{F}(oldsymbol{x})=oldsymbol{0}$ using complex-step derivatives. It develops convergence theory showing that the Jacobian-based method is linear for finite $h$ and becomes quadratic as $h\to 0$, while the Jacobian-free variant achieves quadratic convergence for moderately small $h>0$, with rigorous results supported by numerical experiments. The analysis covers both scalar and system cases, employing fixed-point reformulations and Ostrowski-type arguments, and is complemented by applications to stiff ODE time stepping with a symplectic Runge-Kutta method and to the DNLS equation. Overall, the complex-step Jacobian-free Newton method provides a robust, derivative-free Newton-Krylov approach with predictable quadratic convergence for large-scale nonlinear problems.

Abstract

Considered herein is a modified Newton method for the numerical solution of nonlinear equations where the Jacobian is approximated using a complex-step derivative approximation. We show that this method converges for sufficiently small complex-step values, which need not be infinitesimal. Notably, when the individual derivatives in the Jacobian matrix are approximated using the complex-step method, the convergence is linear and becomes quadratic as the complex-step approaches zero. However, when the Jacobian matrix is approximated by the nonlinear complex-step derivative approximation, the convergence rate remains quadratic for any appropriately small complex-step value, not just in the limit as it approaches zero. This claim is supported by numerical experiments. Additionally, we demonstrate the method's robust applicability in solving nonlinear systems arising from differential equations, where it is implemented as a Jacobian-free Newton-Krylov method.

The complex-step Newton method and its convergence

TL;DR

This work analyzes the complex-step Newton method for solving and using complex-step derivatives. It develops convergence theory showing that the Jacobian-based method is linear for finite and becomes quadratic as , while the Jacobian-free variant achieves quadratic convergence for moderately small , with rigorous results supported by numerical experiments. The analysis covers both scalar and system cases, employing fixed-point reformulations and Ostrowski-type arguments, and is complemented by applications to stiff ODE time stepping with a symplectic Runge-Kutta method and to the DNLS equation. Overall, the complex-step Jacobian-free Newton method provides a robust, derivative-free Newton-Krylov approach with predictable quadratic convergence for large-scale nonlinear problems.

Abstract

Considered herein is a modified Newton method for the numerical solution of nonlinear equations where the Jacobian is approximated using a complex-step derivative approximation. We show that this method converges for sufficiently small complex-step values, which need not be infinitesimal. Notably, when the individual derivatives in the Jacobian matrix are approximated using the complex-step method, the convergence is linear and becomes quadratic as the complex-step approaches zero. However, when the Jacobian matrix is approximated by the nonlinear complex-step derivative approximation, the convergence rate remains quadratic for any appropriately small complex-step value, not just in the limit as it approaches zero. This claim is supported by numerical experiments. Additionally, we demonstrate the method's robust applicability in solving nonlinear systems arising from differential equations, where it is implemented as a Jacobian-free Newton-Krylov method.
Paper Structure (10 sections, 6 theorems, 89 equations, 6 figures)

This paper contains 10 sections, 6 theorems, 89 equations, 6 figures.

Table of Contents

  1. Introduction
  2. Convergence
  3. Scalar equations
  4. Convergence for scalar equations
  5. Experimental convergence rate
  6. Systems of equations
  7. Convergence for systems of equations
  8. Experimental convergence rates
  9. Performance in applications
  10. Conclusions

Key Result

Lemma 2.1

If the function $f$ is real analytic in a closed interval $I$ and $x^\ast$ in the interior of $I$ with $f(x^\ast)=0$ and $f'(x^\ast)\not=0$, then there are $\bar{h}>0$ and $m,M>0$ independent of $h$ such that $m\leq \frac{1}{h}|\mathrm{Im} f(x+ih)|\leq M$ for all $0<h<\bar{h}$.

Figures (6)

  • Figure 1: Convergence rate as a function of $h$ for a scalar function $f$
  • Figure 2: Convergence rate and number of iterations as a function of $h$ of the complex-step Jacobian-free Newton method
  • Figure 3: Numerical solution of a stiff ordinary differential equation obtained by solving the nonlinear systems using the complex-step Jacobian-free Newton method
  • Figure 4: Projection of the spiral orbit of the Olsen system in the space $A-B-X$
  • Figure 5: The numerical solution at $t=0$ and $t=100$
  • ...and 1 more figures

Theorems & Definitions (12)

  • Lemma 2.1
  • proof
  • Theorem 2.1
  • proof
  • Lemma 2.2
  • proof
  • Theorem 2.2
  • proof
  • Lemma 2.3
  • proof
  • ...and 2 more