A fast neural hybrid Newton solver adapted to implicit methods for nonlinear dynamics

TL;DR

This work tackles the costly nonlinear solves at each time step in implicit time-stepping of stiff nonlinear PDEs by introducing a neural hybrid Newton solver. A scheme-informed unsupervised neural time-stepper provides a high-quality initial guess for Newton iterations, reducing iteration counts while preserving the original scheme's stability and structure. The authors establish theoretical bounds on Newton-iteration reduction and generalisation error, and validate the approach on the Allen–Cahn equation in 1D and 2D, showing meaningful speed-ups and robust structure preservation. The methods offer a practical, data-efficient way to accelerate implicit solvers without requiring labeled data, with potential applicability to a broad class of time-evolving PDEs.

Abstract

The use of implicit time-stepping schemes for the numerical approximation of solutions to stiff nonlinear time-evolution equations brings well-known advantages including, typically, better stability behaviour and corresponding support of larger time steps, and better structure preservation properties. However, this comes at the price of having to solve a nonlinear equation at every time step of the numerical scheme. In this work, we propose a novel deep learning based hybrid Newton's method to accelerate this solution of the nonlinear time step system for stiff time-evolution nonlinear equations. We propose a targeted learning strategy which facilitates robust unsupervised learning in an offline phase and provides a highly efficient initialisation for the Newton iteration leading to consistent acceleration of Newton's method. A quantifiable rate of improvement in Newton's method achieved by improved initialisation is provided and we analyse the upper bound of the generalisation error of our unsupervised learning strategy. These theoretical results are supported by extensive numerical results, demonstrating the efficiency of our proposed neural hybrid solver both in one- and two-dimensional cases.

Figures (10)

• Figure 1: The architecture of the proposed implicit-scheme informed neural network. The purple blocks represent the convolution operator and the green blocks represent the activation function.
• Figure 2: (a) Experimental and theoretical Newton iteration count in solving $1$-dimensional Allen--Cahn equation. The blue cross line is the experiment result, while the red dashed line is the theoretical estimation; (b) Experimental and theoretical Newton iteration count in solving $2$-dimensional Allen--Cahn equation. The blue cross line is the experiment result, while the red dashed line is the theoretical estimation.
• Figure 3: (a) Loss curve; (b) Comparison between neural network prediction and exact midpoint solution
• Figure 4: Update size VS iteration count (1D). The yellow line represents iteration with direct initial guess, the blue line represents iteration with NN prediction initial, and the green line represents iteration with ETD solution initial. (a) $\tau_{midpoint}=0.5$; (b) $\tau_{midpoint}=1$; (b) $\tau_{midpoint}=2$.
• Figure 5: (a) CPU time VS the $L^2$ error. The green line represents the performance of ETD scheme, the blue line represents the midpoint method, the black line and the yellow cross are the result of NN hybrid and ETD hybrid solvers, respectively; (b) average CPU time in a single time step when ETD solutions with different time steps are used as the initial guess. The green line shows the average CPU time of ETD scheme, the black and blue lines represent the CPU time of direct initial guess and the NN hybrid method, respectively. Here, we fix $\tau_{midpoint}=1$. (c) $L^2$ error of initial guesses when $\tau_{ETD}$ varies from $0.1$ to $1$. Here, $\tau_{midpoint}=1$.

Theorems & Definitions (35)

• Remark 2.1
• Definition 2.1
• Remark 2.2
• Example 2.1
• Remark 2.3
• Theorem 3.1: Theorem $4.4$ in SuliMayers2003
• Theorem 3.2: Theoretical upper bound of iteration count
• proof
• Remark 3.1
• Remark 3.2