Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Anderson Acceleration For Perturbed Newton Methods

Matt Dallas

TL;DR

This work analyzes Anderson acceleration applied to perturbed Newton methods (pNMs) for nonlinear root-finding, focusing on singular problems where standard Newton convergence is only linear. It establishes local linear convergence for γ-safeguarded AA applied to pNMs under 2-regularity, and extends the results to Levenberg-Marquardt (LM), yielding a novel acceleration framework for LM in singular settings. A key feature is adaptive γ-safeguarding, which scales the Anderson step to preserve or detect superlinear convergence, ensuring robustness across both singular and nonsingular regimes. Numerical experiments on the Chandrasekhar H-equation and incompressible channel flow illustrate practical acceleration and the safeguarding mechanism near bifurcations. Overall, the paper provides a rigorous theory and compelling evidence that γ-safeguarded AA enhances pNMs, including LM, in challenging nonlinear problems.

Abstract

We present a convergence theory for Anderson acceleration (AA) applied to perturbed Newton methods (pNMs) for computing roots of nonlinear problems. Two important special cases are the classical Newton method and the Levenberg-Marquardt method. We prove that if a problem is 2-regular, then Anderson accelerated pNMs coupled with a safeguarding scheme, known as $γ$-safeguarding, converge locally linearly in a starlike domain of convergence, but with an improved rate of convergence compared to standard perturbed Newton methods. Since Levenberg-Marquardt methods are a special case of pNMs, we obtain a novel acceleration and local convergence result for Anderson accelerated Levenberg-Marquardt. We further show that $γ$-safeguarding can detect if the underlying perturbed Newton method is converging superlinearly, and respond by tuning the Anderson step down. We demonstrate the methods on several benchmark problems in the literature.

Anderson Acceleration For Perturbed Newton Methods

TL;DR

This work analyzes Anderson acceleration applied to perturbed Newton methods (pNMs) for nonlinear root-finding, focusing on singular problems where standard Newton convergence is only linear. It establishes local linear convergence for γ-safeguarded AA applied to pNMs under 2-regularity, and extends the results to Levenberg-Marquardt (LM), yielding a novel acceleration framework for LM in singular settings. A key feature is adaptive γ-safeguarding, which scales the Anderson step to preserve or detect superlinear convergence, ensuring robustness across both singular and nonsingular regimes. Numerical experiments on the Chandrasekhar H-equation and incompressible channel flow illustrate practical acceleration and the safeguarding mechanism near bifurcations. Overall, the paper provides a rigorous theory and compelling evidence that γ-safeguarded AA enhances pNMs, including LM, in challenging nonlinear problems.

Abstract

We present a convergence theory for Anderson acceleration (AA) applied to perturbed Newton methods (pNMs) for computing roots of nonlinear problems. Two important special cases are the classical Newton method and the Levenberg-Marquardt method. We prove that if a problem is 2-regular, then Anderson accelerated pNMs coupled with a safeguarding scheme, known as -safeguarding, converge locally linearly in a starlike domain of convergence, but with an improved rate of convergence compared to standard perturbed Newton methods. Since Levenberg-Marquardt methods are a special case of pNMs, we obtain a novel acceleration and local convergence result for Anderson accelerated Levenberg-Marquardt. We further show that -safeguarding can detect if the underlying perturbed Newton method is converging superlinearly, and respond by tuning the Anderson step down. We demonstrate the methods on several benchmark problems in the literature.

Paper Structure

This paper contains 14 sections, 5 theorems, 23 equations, 7 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Perturbed Newton Methods
  3. Anderson Acceleration and Safeguarding
  4. Anderson Acceleration
  5. Adaptive $\gamma$-Safeguarding
  6. Convergence Analysis
  7. Linear Convergence
  8. Superlinear Convergence
  9. Special Case: Levenberg-Marquardt
  10. Numerical Experiments
  11. The Chandrasekhar H-Equation
  12. Incompressible Channel Flow
  13. AALM Under Local Error Bound
  14. Conclusion

Key Result

Lemma 2.1

IzKuSo18-1 Suppose that $f$ is $C^2$ and $f"(x)-f"(x^*) = \mathcal{O}(\|x-x^*\|)$ as $x\to x^*$, and let $f(x)$ be 2-regular at $x^*$ along some some vector $v^*$ of unit length. Let $\Omega(x) = \mathcal{O}(\|x-x^*\|)$ as $x\to x^*$, and suppose that for every $\Delta>0$, there is a $\varepsilon>0$

Figures (7)

  • Figure 1: Channel flow domain and boundaries. The red vertical segment is $\Gamma_{\text{in}}$, the blue is $\Gamma_{\text{out}}$, and the remaining boundaries, in black, are $\Gamma_{\text{wall}}$.
  • Figure 2: Stable velocity solutions for different values of $\mu$. The top and bottom figures are stable solutions when $\mu=0.9$. When $\mu=1$, the middle solution is stable.
  • Figure 3: Residual history for channel flow problem with $\mu = 0.98$. Left: results for Newton and Newton--Anderson with algorithmic depths $m=1$, 2, 5, and 10. Right: results for $\gamma$-safeguarded Newton--Anderson applied asymptotically plotted with dash-dotted lines. We set $r=0$ in Algorithm \ref{['alg:adapgsg']}.
  • Figure 4: Residual history for channel flow problem with $\mu = 0.94$. Left: results for Newton and Newton--Anderson with algorithmic depths $m=1$, 2, 5, and 10. Right: results for $\gamma$-safeguarded Newton--Anderson applied asymptotically plotted with dash-dotted lines. We set $r=0$ in Algorithm \ref{['alg:adapgsg']}.
  • Figure 5: Left: Residual history for test function Beh1. Right: Residual history for test function Beh2. Dash-dotted lines correspond to AALM with $\gamma$-safeguarding. Note that $\gamma$AALM(1,0.9) and $\gamma$AALM(5,0.9) coincide in each of these experiments.
  • ...and 2 more figures

Theorems & Definitions (6)

  • Lemma 2.1
  • Theorem 4.1
  • Lemma 4.1
  • Theorem 4.2
  • proof
  • Corollary 4.1