Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RandNet-Parareal: a time-parallel PDE solver using Random Neural Networks

Guglielmo Gattiglio, Lyudmila Grigoryeva, Massimiliano Tamborrino

TL;DR

RandNet-Parareal is introduced, a novel method to learn the discrepancy between the coarse and fine solutions using random neural networks (RandNets) and achieves speed gains up to x125 and x22 compared to the fine solver run serially and Parareal, respectively.

Abstract

Parallel-in-time (PinT) techniques have been proposed to solve systems of time-dependent differential equations by parallelizing the temporal domain. Among them, Parareal computes the solution sequentially using an inaccurate (fast) solver, and then "corrects" it using an accurate (slow) integrator that runs in parallel across temporal subintervals. This work introduces RandNet-Parareal, a novel method to learn the discrepancy between the coarse and fine solutions using random neural networks (RandNets). RandNet-Parareal achieves speed gains up to x125 and x22 compared to the fine solver run serially and Parareal, respectively. Beyond theoretical guarantees of RandNets as universal approximators, these models are quick to train, allowing the PinT solution of partial differential equations on a spatial mesh of up to $10^5$ points with minimal overhead, dramatically increasing the scalability of existing PinT approaches. RandNet-Parareal's numerical performance is illustrated on systems of real-world significance, such as the viscous Burgers' equation, the Diffusion-Reaction equation, the two- and three-dimensional Brusselator, and the shallow water equation.

RandNet-Parareal: a time-parallel PDE solver using Random Neural Networks

TL;DR

RandNet-Parareal is introduced, a novel method to learn the discrepancy between the coarse and fine solutions using random neural networks (RandNets) and achieves speed gains up to x125 and x22 compared to the fine solver run serially and Parareal, respectively.

Abstract

Parallel-in-time (PinT) techniques have been proposed to solve systems of time-dependent differential equations by parallelizing the temporal domain. Among them, Parareal computes the solution sequentially using an inaccurate (fast) solver, and then "corrects" it using an accurate (slow) integrator that runs in parallel across temporal subintervals. This work introduces RandNet-Parareal, a novel method to learn the discrepancy between the coarse and fine solutions using random neural networks (RandNets). RandNet-Parareal achieves speed gains up to x125 and x22 compared to the fine solver run serially and Parareal, respectively. Beyond theoretical guarantees of RandNets as universal approximators, these models are quick to train, allowing the PinT solution of partial differential equations on a spatial mesh of up to points with minimal overhead, dramatically increasing the scalability of existing PinT approaches. RandNet-Parareal's numerical performance is illustrated on systems of real-world significance, such as the viscous Burgers' equation, the Diffusion-Reaction equation, the two- and three-dimensional Brusselator, and the shallow water equation.

Paper Structure

This paper contains 19 sections, 1 theorem, 29 equations, 8 figures, 8 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. The Parareal algorithm
  3. GParareal and Nearest Neighbors GParareal
  4. Random neural networks Parareal (RandNets-Parareal)
  5. Numerical Experiments
  6. Viscous Burgers' equation
  7. Diffusion-Reaction system
  8. Shallow water equation
  9. Discussion and limitations
  10. Pseudocodes
  11. Additional details on the nnGParareal correction function
  12. Computational complexity analysis
  13. Robustness study
  14. Additional numerical experiments: 2D and 3D Brusselator PDE
  15. Accuracy and runtimes across models and algorithms
  16. ...and 4 more sections

Key Result

Proposition 1

Let $H^*: \mathbb{R}^d \rightarrow \mathbb{R}$, $\boldsymbol{U} \longmapsto H^*(\boldsymbol{U})$ be an unknown function we wish to approximate with $H_W^{A,\boldsymbol{\zeta}}$ defined in HA. Suppose $H^*$ can be represented as $H^*(\boldsymbol{U})=\int_{\mathbb{R}^d} e^{i\langle\mathbf{w}, \boldsym and for any $\delta \in(0,1)$, the random neural network $H_{{W}}^{{A}, \boldsymbol{\zeta}}$ satisf

Figures (8)

  • Figure 1: Speed-ups (left) and runtimes (right) of Parareal, nnGParareal ($m_{\rm nnGP}$=$20$), and RandNet-Parareal ($m_{\rm RandNet}\text{=}4$, $M\text{=}100$) for the two-dimensional Diffusion-Reaction system versus the number $d$ of dimensions (bottom x-axis) and $N$ cores (top x-axis) capped at $512$ to simulate limited resources.
  • Figure 2: Numerical solution of the SWE for $(x,y) \in [-5,5]\times[0,5]$ with $N_x=264$ and $N_y=133$ for a range of system times $t$. Only the water depth $h$ (blue) is plotted.
  • Figure 3: Theoretical model cost (panel A) and theoretical total cost (panel B), as functions of the dimension $d$ (and the corresponding $N$). The results are reported in terms of $\log_{10}({\rm hours})$.
  • Figure 4: Histogram of the iterations to convergence $K_{\rm RandNet\text{-}Para}$ of RandNet-Parareal for $d=128$ for Burgers' equation. We sample the network weights $A$, $\boldsymbol{\zeta}$ 100 times. For each set of weights, we run RandNet-Parareal for $m_{\rm RandNet} \in \{2, 3,\ldots, 20\}$ and $M \in \{20, 30, 40, \ldots, 500\}$. The left and right panels show the aggregated histograms of $K_{\rm RandNet\text{-}Para}$ versus $m_{\rm RandNet}$ and $M$, respectively.
  • Figure 5: Histogram of the iterations to convergence $K_{\rm RandNet\text{-}Para}$ of RandNet-Parareal for $d=722$ for Diffusion-Reaction equation. We sample the network weights $A$, $\boldsymbol{\zeta}$ 100 times. For each set of weights, we run RandNet-Parareal for $m_{\rm RandNet} \in \{2, 3,\ldots, 20\}$ and $M \in \{20, 30, 40, \ldots, 500\}$. The left and right panels show the aggregated histograms of $K_{\rm RandNet\text{-}Para}$ versus $m_{\rm RandNet}$ and $M$, respectively.
  • ...and 3 more figures

Theorems & Definitions (1)

  • Proposition 1: Approximation bound, gonon2023approximation, Proposition 3