Learning phase-space flows using time-discrete implicit Runge-Kutta PINNs

TL;DR

A computational framework for obtaining multidimensional phase-space solutions of systems of non-linear coupled differential equations, using high-order implicit Runge-Kutta Physics- Informed Neural Networks (IRK-PINNs) schemes, which is particularly useful for explicitly time-independent and periodic fields.

Abstract

We present a computational framework for obtaining multidimensional phase-space solutions of systems of non-linear coupled differential equations, using high-order implicit Runge-Kutta Physics- Informed Neural Networks (IRK-PINNs) schemes. Building upon foundational work originally solving differential equations for fields depending on coordinates [J. Comput. Phys. 378, 686 (2019)], we adapt the scheme to a context where the coordinates are treated as functions. This modification enables us to efficiently solve equations of motion for a particle in an external field. Our scheme is particularly useful for explicitly time-independent and periodic fields. We apply this approach to successfully solve the equations of motion for a mass particle placed in a central force field and a charged particle in a periodic electric field.

Figures (4)

• Figure 1: Calculated trajectories of a particle in a central Gaussian potential, with potential depth $V_0 = 10$ and extension $a = \sqrt{5}$. The IRK-PINN and numerically exact solutions are plotted with blue and red colours, respectively. Panel (a) shows the entire training phase space consisting of 2 000 points. Due to the complex distribution of the solutions with respect to the initial position, a big set of training data is needed. Panel (b) shows the result of the low-order IRK method (blue) on the validation set, consisting of 100 points, for visual clarity. In addition, the PINN guess on parameter initialisation is displayed (red). Panels (c) and (d) show the IRK-PINN predictions on the validation set for the time step $\Delta t$ and $2 \Delta t$, respectively. The IRK-PINNs approach adopted a Runge-Kutta order $q=100$, time step $\Delta t = 0.8$, dense MLP containing 5 hidden layers with 200 nodes each, and a bipolar sigmoid as the activation function. The training was carried out for 100 000 epochs with the ADAM optimiser followed by 40 000 epochs with L-BFGS-B. Total $L^1\text{-error}=0.58~\%$.
• Figure 2: Accuracy of IRK-PINNs method for different Runge-Kutta orders $q=30,100,499$ in solving the trajectories of a charged particle in periodic electric field, as described by \ref{['eq:charged_particle_diff_eq']} with the parameters $\alpha=0.5$ and $A=10$. Accuracy is quantified using the $L^1$-error calculated on a validation set of 2 500 points. Training involved 50 000 epochs each with the ADAM and L-FBGS-B optimizers. The model architecture included an MLP with 5 hidden layers, each composing 32 nodes, and employed the $SiLU(x)$ activation function. The training utilized 5 000 points.
• Figure 3: Fit to the heat equation data with IRK-PINNs of order $q=100$ and time step $\Delta t = 0.7$, compared to the analytical solution of the problem. The training was performed for 25000 epochs with ADAM, 40000 epochs with L-BFGS-B, using an MLP with 20 hidden layers with 32 nodes each and $SiLU$ as activation function. (a) The initial configuration space, consisting of $N=500$ points. (b) The predition of the PINN on parameter inisialization. This initial prediction is totally arbitrary and far from the correct result, which shows the robustness of the learning process. (c) The analytical solution of the heat equation after one time step. (d) The prediction of the IRK-PINN after training. Total $L^1$ error $=4.81\%$
• Figure 4: Fit to the Taylor-Green vortex data with $\nu = 1$, $\rho=2$ using a IRK-PINN of order $q=100$ and time step $\Delta t = 1$. The training was carried out for 20000 epochs with ADAM and 20000 epochs with L-BFGS-B, using an MLP with 10 hidden layers with 16 nodes each and $SiLU$ activation function, and a sampling of $N=300$ points. (a) and (b) The results of the trained neural network prediction for $u(t_{n+1},x)$ and $v(t_{n+1},x)$, respectively. (c) and (d) Deviation from the analytical solution in \ref{['eq:TaylorVortex']}. Total $L^1$-error $=2.01\%$