Runge-Kutta Physics Informed Neural Networks: Formulation and Analysis

TL;DR

This work addresses training neural network surrogates for time-dependent PDEs while preserving stability and qualitative dynamics. It introduces Runge-Kutta Physics Informed Neural Networks (RK-PINNs), designing the discrete loss from a time-discrete framework and embedding Runge-Kutta or time-Galerkin discretizations into the training objective to inherit stability properties. The authors derive maximal regularity (MR) estimates for B-stable RK schemes and for both continuous and discontinuous Galerkin time discretizations, providing energy-based proofs and establishing convergence of discrete minimizers to the PDE solution in $L^2((0,T);H^1(\Omega))$, even with variable time steps. The theory is complemented by numerical experiments on linear parabolic and wave equations, showing higher accuracy, conservation properties, and the ability to tailor the qualitative behavior of the discrete solution through the choice of time discretization. Overall, the RK-PINN framework offers a principled, high-order, and physically consistent approach to learning PDE solutions in strong mathematical settings.

Abstract

In this paper we consider time-dependent PDEs discretized by a special class of Physics Informed Neural Networks whose design is based on the framework of Runge--Kutta and related time-Galerkin discretizations. The primary motivation for using such methods is that alternative time-discrete schemes not only enable higher-order approximations but also have a crucial impact on the qualitative behavior of the discrete solutions. The design of the methods follows a novel training approach based on two key principles: (a) the discrete loss is designed using a time-discrete framework, and (b) the final loss formulation incorporates Runge--Kutta or time-Galerkin discretization in a carefully structured manner. We then demonstrate that the resulting methods inherit the stability properties of the Runge--Kutta or time-Galerkin schemes, and furthermore, their computational behavior aligns with that of the original time discrete method used in their formulation. In our analysis, we focus on linear parabolic equations, demonstrating both the stability of the methods and the convergence of the discrete minimizers to solutions of the underlying evolution PDE. An important novel aspect of our work is the derivation of maximal regularity (MR) estimates for B-stable Runge--Kutta schemes and both continuous and discontinuous Galerkin time discretizations. This allows us to provide new energy-based proofs for maximal regularity estimates previously established by Kovács, Li, and Lubich, now in the Hilbert space setting and with the flexibility of variable time steps.

Figures (7)

• Figure 1: Absolute misfit across various schemes
• Figure 2: Heat conservation (plot of $H(t)=\int _{\varOmega} u (x, t )\, {\mathrm d} x$) across various schemes.
• Figure 3: Absolute misfit across various iterative schemes
• Figure 4: Heat conservation across various iterative schemes.
• Figure 5: Smoothing effect for the heat equation. As expected, Gauss and Lobatto methods have oscillating behavior close to initial times for discontinuous data. The full smoothing effect of Radau methods is evident.

Theorems & Definitions (16)

• Remark 1
• Proposition 2
• Lemma 3: $\liminf$ inequality
• Lemma 4: $\limsup$ inequality
• Remark 5: On the abstract assumptions on $\widehat{\varPi} _q ,$ $\varPi _{q-1}$
• Lemma 6: Aubin--Lions Lemma
• Theorem 7: Convergence
• Theorem 8: Maximal $L^2$ regularity for methods of the form \ref{['eq:nm-abstr1']}
• Proposition 9: Maximal $L^2$ regularity of cG methods
• Proposition 10: Maximal $L^2$ regularity of dG methods