Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Parallel-in-Time Solutions with Random Projection Neural Networks

Marta M. Betcke, Lisa Maria Kreusser, Davide Murari

TL;DR

The paper tackles accelerating time integration for initial-value problems by embedding Random Projection Neural Networks as the coarse propagator in the Parareal framework. It provides a quadrature-based a-posteriori error bound for neural solvers, ensuring convergence guarantees carry over to the NN-assisted Parareal method, and shows online training of RPNNs per subinterval to adapt to local dynamics. Empirical results on SIR, ROBER, Lorenz, Arenstorf, and Burgers demonstrate competitive accuracy with substantial speedups and without offline training, highlighting the method’s practicality for stiff and chaotic systems. Overall, the work offers a theoretically grounded, computationally efficient hybrid solver that leverages low-cost, expressivity-rich RPNN coarse propagators to enable scalable parallel-in-time simulations.

Abstract

This paper considers one of the fundamental parallel-in-time methods for the solution of ordinary differential equations, Parareal, and extends it by adopting a neural network as a coarse propagator. We provide a theoretical analysis of the convergence properties of the proposed algorithm and show its effectiveness for several examples, including Lorenz and Burgers' equations. In our numerical simulations, we further specialize the underpinning neural architecture to Random Projection Neural Networks (RPNNs), a 2-layer neural network where the first layer weights are drawn at random rather than optimized. This restriction substantially increases the efficiency of fitting RPNN's weights in comparison to a standard feedforward network without negatively impacting the accuracy, as demonstrated in the SIR system example.

Parallel-in-Time Solutions with Random Projection Neural Networks

TL;DR

The paper tackles accelerating time integration for initial-value problems by embedding Random Projection Neural Networks as the coarse propagator in the Parareal framework. It provides a quadrature-based a-posteriori error bound for neural solvers, ensuring convergence guarantees carry over to the NN-assisted Parareal method, and shows online training of RPNNs per subinterval to adapt to local dynamics. Empirical results on SIR, ROBER, Lorenz, Arenstorf, and Burgers demonstrate competitive accuracy with substantial speedups and without offline training, highlighting the method’s practicality for stiff and chaotic systems. Overall, the work offers a theoretically grounded, computationally efficient hybrid solver that leverages low-cost, expressivity-rich RPNN coarse propagators to enable scalable parallel-in-time simulations.

Abstract

This paper considers one of the fundamental parallel-in-time methods for the solution of ordinary differential equations, Parareal, and extends it by adopting a neural network as a coarse propagator. We provide a theoretical analysis of the convergence properties of the proposed algorithm and show its effectiveness for several examples, including Lorenz and Burgers' equations. In our numerical simulations, we further specialize the underpinning neural architecture to Random Projection Neural Networks (RPNNs), a 2-layer neural network where the first layer weights are drawn at random rather than optimized. This restriction substantially increases the efficiency of fitting RPNN's weights in comparison to a standard feedforward network without negatively impacting the accuracy, as demonstrated in the SIR system example.
Paper Structure (27 sections, 7 theorems, 72 equations, 8 figures, 8 tables, 1 algorithm)

This paper contains 27 sections, 7 theorems, 72 equations, 8 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Contributions
  3. Outline
  4. Parareal method
  5. The method
  6. Interpretation of the correction term
  7. Convergence
  8. A-posteriori error estimate for neural network-based solvers
  9. Parareal method based on Random Projection Neural Networks
  10. Architecture design
  11. Algorithm design
  12. Training strategy
  13. Implementation details
  14. Convergence of the RPNN-based Parareal method
  15. Numerical results
  16. ...and 12 more sections

Key Result

Theorem 1

Let us consider the initial value problem eq:ode and partition the time interval $[0,T]$ into $N$ intervals of size $\Delta t = T/N$ using a grid of nodes $t_n=n\Delta t$. Assume that the fine integrator $\varphi^{{\Delta t}}_F$ coincides with the exact flow map $\phi_{\mathcal{F}}^{\Delta t}$, i.e. for every $\mathbf{x}\in\mathbb{R}^d$, and also that there exists $\beta>0$ such that for every $\

Figures (8)

  • Figure 1: SIR: Hybrid Parareal solution with (left) a RPNN-based coarse propagator, (right) flow map coarse propagator.
  • Figure 2: ROBER: Components of the Hybrid Parareal solution. To plot all components on the same scale, $\mathbf{x}_2$ was scaled by a factor of $10^4$.
  • Figure 3: Lorenz: Hybrid Parareal solution with (left) uniform collocation points, (right) Lobatto collocation points.
  • Figure 4: Arenstorf: Components of the Hybrid Parareal solution (left), and the orbit of the initial condition (right).
  • Figure 5: Burgers: Snapshots of the solution obtained with Hybrid Parareal (left), comparison of the solution surfaces between Hybrid Parareal and the fine integrator applied serially (right). Solution corresponding to $u_0(x)=\sin{(2\pi x)}$.
  • ...and 3 more figures

Theorems & Definitions (11)

  • Theorem 1: Theorem 1 in gander2008nonlinear
  • Theorem 2: Quadrature-based a-posteriori error estimate
  • Theorem 3: Gröbner-Alekseev
  • proof : Proof of Theorem \ref{['thm:QuadAposteriori']}
  • Theorem 4: Existence and regularity of the solution
  • proof
  • Proposition 1: Convergence of the hybrid Parareal method
  • proof
  • Lemma 1
  • proof
  • ...and 1 more