Improved Training of Physics-Informed Neural Networks with Model Ensembles
Katsiaryna Haitsiukevich, Alexander Ilin
TL;DR
This work tackles the instability of physics-informed neural networks (PINNs) by training an ensemble of PINNs and using their agreement to guide a gradual, data-informed expansion of the solution domain, treating time and space on equal footing without a pre-defined time schedule. The approach leverages ensemble median and variance to decide where to include new collocation points and, optionally, to create pseudo-labels that strengthen supervision, addressing the tendency of single PINNs to converge to wrong solutions. Empirical results on convection, reaction, and diffusion-type PDEs demonstrate that the ensemble method stabilizes training and achieves competitive accuracy compared with time-adaptive PINN variants, while offering a mechanism for uncertainty quantification via ensemble variability. The method provides a flexible, data-friendly framework for solving PDEs with PINNs, potentially benefiting applications with irregular observation layouts and enabling parallelizable training workflows.
Abstract
Learning the solution of partial differential equations (PDEs) with a neural network is an attractive alternative to traditional solvers due to its elegance, greater flexibility and the ease of incorporating observed data. However, training such physics-informed neural networks (PINNs) is notoriously difficult in practice since PINNs often converge to wrong solutions. In this paper, we address this problem by training an ensemble of PINNs. Our approach is motivated by the observation that individual PINN models find similar solutions in the vicinity of points with targets (e.g., observed data or initial conditions) while their solutions may substantially differ farther away from such points. Therefore, we propose to use the ensemble agreement as the criterion for gradual expansion of the solution interval, that is including new points for computing the loss derived from differential equations. Due to the flexibility of the domain expansion, our algorithm can easily incorporate measurements in arbitrary locations. In contrast to the existing PINN algorithms with time-adaptive strategies, the proposed algorithm does not need a pre-defined schedule of interval expansion and it treats time and space equally. We experimentally show that the proposed algorithm can stabilize PINN training and yield performance competitive to the recent variants of PINNs trained with time adaptation.