Variational Physics-Informed Neural Networks For Solving Partial Differential Equations

TL;DR

The paper introduces variational physics-informed neural networks (VPINN), a Petrov-Galerkin framework that replaces the PINN strong-form residual with a variational (weak) residual. By selecting neural networks as the trial space and flexible test spaces (e.g., sine functions or Legendre polynomials), VPINN leverages integration by parts to lower the differential order and reduce training cost while improving accuracy. The authors derive analytic variational residuals for shallow networks and demonstrate substantial performance gains over PINNs on 1D Burgers and Poisson problems, and extend the approach to deep networks using Gauss-type quadrature. Through 1D and 2D numerical experiments, VPINN shows higher accuracy and faster convergence, highlighting its potential for efficient PDE solving with neural networks and offering directions for future quadrature-based analyses and domain-decomposed learning.

Abstract

Physics-informed neural networks (PINNs) [31] use automatic differentiation to solve partial differential equations (PDEs) by penalizing the PDE in the loss function at a random set of points in the domain of interest. Here, we develop a Petrov-Galerkin version of PINNs based on the nonlinear approximation of deep neural networks (DNNs) by selecting the {\em trial space} to be the space of neural networks and the {\em test space} to be the space of Legendre polynomials. We formulate the \textit{variational residual} of the PDE using the DNN approximation by incorporating the variational form of the problem into the loss function of the network and construct a \textit{variational physics-informed neural network} (VPINN). By integrating by parts the integrand in the variational form, we lower the order of the differential operators represented by the neural networks, hence effectively reducing the training cost in VPINNs while increasing their accuracy compared to PINNs that essentially employ delta test functions. For shallow networks with one hidden layer, we analytically obtain explicit forms of the \textit{variational residual}. We demonstrate the performance of the new formulation for several examples that show clear advantages of VPINNs over PINNs in terms of both accuracy and speed.

Figures (10)

• Figure 1: \newlabelFig: VPINN0 Schematic of VPINN in a Petrov-Galerkin formulation. Trial functions belong to the space of NN and test functions can be chosen from a separate NN or other function spaces such as polynomials and trigonometric functions. Red color represents the differential operators on trial space. Green color represents the test functions and their derivatives. Blue color represents the variational residuals $\mathcal{R}$.
• Figure 2: \newlabelFig: St Burger0 One-dimensional steady state Burger's equation: VPINN with $\mathcal{R}^{(1)} = \mathcal{R}^{(2)}$ formulation. Left: exact solution $\sin(2.1 \pi x)$ and VPINN approximation. Right: point-wise error averaged over several random network initializations. See Table \ref{['Table: VPINN para burger']} for VPINN hyperparameters.
• Figure 3: \newlabelFig: St Burger compare0 One-dimensional steady state Burger's equation: network initialization effect on optimization performance. Left: comparison of loss values. Right: comparison of point-wise error. The red dashed line shows a more successful optimization and thus a much lower error. See Table \ref{['Table: VPINN para burger']} for VPINN hyperparameters.
• Figure 4: \newlabelFig: St Burger VR3 Lbw compare0 One-dimensional steady state Burger's equation: effect of penalty parameter $\tau$ in the $\mathcal{R}^{(3)}$ formulation. Top Left: comparison of exact solution $\sin(2.1 \pi x)$ and VPINN approximation for different $\tau$. Top Right: loss value. Bottom: the zoom-in display of point-wise error close to boundaries. See Table \ref{['Table: VPINN para burger']} for VPINN hyperparameters.
• Figure 5: \newlabelFig: St Burger Vanishing Bound NK compare0 Example \ref{['Ex: STBurger - vanishing boundary']}. One-dimensional steady state Burger's equation: error convergence by increasing $N$ (number of neurons) and $K$ (number of test functions). Shallow network with $L=1$ hidden layer, sine activation, and sine test function.

Theorems & Definitions (7)

• Remark 3.1: Analogy between strong-form and variational residual
• Remark 4.1
• Example 4.2
• Example 4.3
• Example 4.4
• Example 5.1: One-Dimensional Poisson's Equation
• Example 5.2: Two-Dimensional Poisson's Equation