Learning the Exact Time Integration Algorithm for Initial Value Problems by Randomized Neural Networks
Suchuan Dong, Naxian Ni
TL;DR
The paper presents a physics-informed, ELM-based framework to learn the exact time integration algorithm for initial value problems by training a high-dimensional algorithmic function $\psi(y_0,t_0,ξ)$ that satisfies $\dfrac{∂ψ}{∂ξ}=f(ψ,t_0+ξ)$. By representing $ψ$ with randomized NNs and solving the associated PDE system via nonlinear least squares, the authors produce learned time-stepping schemes that are explicitly or implicitly defined and usable for arbitrary initial data within a training domain. Key innovations include exploiting periodicity in the RHS to reduce training domains, applying domain decomposition for stiff problems, and deriving detailed Jacobian and residual computations for Gauss-Newton training. Across linear, nonlinear, and chaotic benchmark problems, the learned algorithms exhibit near-exponential error decay with neural degrees of freedom and competitive, often superior, performance relative to traditional adaptive solvers such as DOP853, RK45, Radau, and BDF. The work demonstrates a robust, generalizable approach to learned time integration with potential for efficient long-time simulations in scientific computing.
Abstract
We present a method leveraging extreme learning machine (ELM) type randomized neural networks (NNs) for learning the exact time integration algorithm for initial value problems (IVPs). The exact time integration algorithm for non-autonomous systems can be represented by an algorithmic function in higher dimensions, which satisfies an associated system of partial differential equations with corresponding boundary conditions. Our method learns the algorithmic function by solving this associated system using ELM with a physics informed approach. The trained ELM network serves as the learned algorithm and can be used to solve the IVP with arbitrary initial data or step sizes from some domain. When the right hand side of the non-autonomous system exhibits a periodicity with respect to any of its arguments, while the solution itself to the problem is not periodic, we show that the algorithmic function is either periodic, or when it is not, satisfies a well-defined relation for different periods. This property can greatly simplify the algorithm learning in many problems. We consider explicit and implicit NN formulations, leading to explicit or implicit time integration algorithms, and discuss how to train the ELM network by the nonlinear least squares method. Extensive numerical experiments with benchmark problems, including non-stiff, stiff and chaotic systems, show that the learned NN algorithm produces highly accurate solutions in long-time simulations, with its time-marching errors decreasing nearly exponentially with increasing degrees of freedom in the neural network. We compare extensively the computational performance (accuracy vs.~cost) between the current NN algorithm and the leading traditional time integration algorithms. The learned NN algorithm is computationally competitive, markedly outperforming the traditional algorithms in many problems.