Domain decomposition architectures and Gauss-Newton training for physics-informed neural networks

TL;DR

The paper tackles the training difficulty of physics-informed neural networks for PDEs, driven by spectral bias that hampers high-frequency components. It proposes a combination of finite-basis PINNs (FBPINN) with overlapping domain decomposition and Gauss-Newton (GN) training to accelerate convergence, leveraging a block-sparse energy Gram matrix that arises from localization. The approach yields substantial accuracy and convergence benefits on canonical problems, with 1D and 2D tests showing order-of-magnitude improvements over Adam-based training and demonstrating the practicality of GN for neural PDE solvers. The results highlight the potential of localized, GN-optimized PINNs for efficient, mesh-free PDE solvers in complex settings, aided by the resulting sparsity in the GN system.

Abstract

Approximating the solutions of boundary value problems governed by partial differential equations with neural networks is challenging, largely due to the difficult training process. This difficulty can be partly explained by the spectral bias, that is, the slower convergence of high-frequency components, and can be mitigated by localizing neural networks via (overlapping) domain decomposition. We combine this localization with the Gauss-Newton method as the optimizer to obtain faster convergence than gradient-based schemes such as Adam; this comes at the cost of solving an ill-conditioned linear system in each iteration. Domain decomposition induces a block-sparse structure in the otherwise dense Gauss-Newton system, reducing the computational cost per iteration. Our numerical results indicate that combining localization and Gauss-Newton optimization is promising for neural network-based solvers for partial differential equations.

Figures (5)

• Figure 1: Five overlapping partition of unity window functions $\omega_k$ on $[-1, 1]$ built from cosine (Hann) windows on overlapping subintervals.
• Figure 2: Comparison of prediction in orange against the analytical solution in green (top) and training loss (bottom) of three methods for the ordinary differential equation problem: (left) vanilla PINN, (middle) FBPINN, and (right) Gauss--Newton PINN. Left panels show the loss on a logarithmic scale versus iteration; right panels overlay the learned solution with the exact solution. The green curve shows the exact solution, and the red curve shows the prediction.
• Figure 3: Training loss for the 2D Helmholtz problem: Adam (left) and Gauss--Newton (right).
• Figure 4: Exact solution (left) and absolute errors for Adam (middle) and Gauss--Newton (right).
• Figure 5: Exemplary sparisity pattern of the Gramian $G$ for the FBPINN architecture for the one dimensional ODE problem with $8$ subdomains.