Competitive Physics Informed Networks

TL;DR

This paper addresses the limited accuracy of physics-informed neural networks (PINNs) for solving PDEs due to ill-conditioning from squared residuals. It introduces competitive PINNs (CPINNs), which model a zero-sum game between a PDE solver (PINN) and a discriminator that identifies PDE violations, thereby avoiding the conditioning penalties inherent to PINNs. The authors demonstrate dramatic accuracy gains on the Poisson equation (relative $L_2$ error down to $\sim 10^{-8}$) and substantial improvements on nonlinear problems like the nonlinear Schrödinger equation and Burgers' equation, with CPINNs achieving or approaching single-precision accuracy in practice. They also discuss connections to saddle-point methods, show the critical role of adaptive optimization (ACGD) for CPINNs, and outline future directions to reduce computational overhead while extending the approach to broader constrained learning problems.

Abstract

Neural networks can be trained to solve partial differential equations (PDEs) by using the PDE residual as the loss function. This strategy is called "physics-informed neural networks" (PINNs), but it currently cannot produce high-accuracy solutions, typically attaining about $0.1\%$ relative error. We present an adversarial approach that overcomes this limitation, which we call competitive PINNs (CPINNs). CPINNs train a discriminator that is rewarded for predicting mistakes the PINN makes. The discriminator and PINN participate in a zero-sum game with the exact PDE solution as an optimal strategy. This approach avoids squaring the large condition numbers of PDE discretizations, which is the likely reason for failures of previous attempts to decrease PINN errors even on benign problems. Numerical experiments on a Poisson problem show that CPINNs achieve errors four orders of magnitude smaller than the best-performing PINN. We observe relative errors on the order of single-precision accuracy, consistently decreasing with each epoch. To the authors' knowledge, this is the first time this level of accuracy and convergence behavior has been achieved. Additional experiments on the nonlinear Schrödinger, Burgers', and Allen-Cahn equation show that the benefits of CPINNs are not limited to linear problems.

Figures (6)

• Figure 1: Comparison of CPINN and PINN on the Poisson problem of equation \ref{['e:poisson']} in terms of relative error. CPINN has a faster convergence rate and reduces the $L_2$ error to $1.7\times 10^{-8}$, whereas the PINN case has an $L_2$ error of $1.2 \times 10^{-4}$ even with a larger computational budget.
• Figure 2: (a) Exact solution $u$ to equation \ref{['e:pde']}, absolute errors of (b) PINN + Adam after $3\times 10^7$ training iterations and (c) CPINN + ACGD after $48\,000$ training iterations, and (d) the discriminator.
• Figure 3: Comparison of CPINN and PINN on the nonlinear Schrödinger equation \ref{['SchrodingerPDE']} in terms of relative errors. After $200\,000$ training iterations, PINN cannot reduce the $L_2$ error further, plateauing about $4 \times 10^{-3}$ , whereas CPINN reduces the error to $6\times 10^{-4}$ under a smaller computational budget.
• Figure 4: The relative errors of CPINNs (current) and PINNs on the Burgers' equation.
• Figure 5: Relative error for the Allen--Cahn equation. A CPINN and a PINN that uses the curriculum learning approach of wight2020solving are shown. CPINN does not outperform this particular PINN, which may be due to the overall low accuracy of both methods.