Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Ensemble learning for Physics Informed Neural Networks: a Gradient Boosting approach

Zhiwei Fang, Sifan Wang, Paris Perdikaris

TL;DR

A new training paradigm referred to as "gradient boosting"(GB), which significantly enhances the performance of physics informed neural networks (PINNs), rather than learning the solution of a given PDE using a single neural network directly, employs a sequence of neural networks to achieve a superior outcome.

Abstract

While the popularity of physics-informed neural networks (PINNs) is steadily rising, to this date, PINNs have not been successful in simulating multi-scale and singular perturbation problems. In this work, we present a new training paradigm referred to as "gradient boosting" (GB), which significantly enhances the performance of physics informed neural networks (PINNs). Rather than learning the solution of a given PDE using a single neural network directly, our algorithm employs a sequence of neural networks to achieve a superior outcome. This approach allows us to solve problems presenting great challenges for traditional PINNs. Our numerical experiments demonstrate the effectiveness of our algorithm through various benchmarks, including comparisons with finite element methods and PINNs. Furthermore, this work also unlocks the door to employing ensemble learning techniques in PINNs, providing opportunities for further improvement in solving PDEs.

Ensemble learning for Physics Informed Neural Networks: a Gradient Boosting approach

TL;DR

A new training paradigm referred to as "gradient boosting"(GB), which significantly enhances the performance of physics informed neural networks (PINNs), rather than learning the solution of a given PDE using a single neural network directly, employs a sequence of neural networks to achieve a superior outcome.

Abstract

While the popularity of physics-informed neural networks (PINNs) is steadily rising, to this date, PINNs have not been successful in simulating multi-scale and singular perturbation problems. In this work, we present a new training paradigm referred to as "gradient boosting" (GB), which significantly enhances the performance of physics informed neural networks (PINNs). Rather than learning the solution of a given PDE using a single neural network directly, our algorithm employs a sequence of neural networks to achieve a superior outcome. This approach allows us to solve problems presenting great challenges for traditional PINNs. Our numerical experiments demonstrate the effectiveness of our algorithm through various benchmarks, including comparisons with finite element methods and PINNs. Furthermore, this work also unlocks the door to employing ensemble learning techniques in PINNs, providing opportunities for further improvement in solving PDEs.
Paper Structure (18 sections, 9 equations, 7 figures, 5 tables, 1 algorithm)

This paper contains 18 sections, 9 equations, 7 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Gradient boosting physics informed neural networks
  3. Numerical Experiments
  4. 1D singular perturbation
  5. 2D singular perturbation with boundary layers
  6. 2D singular perturbation with an interior boundary layer
  7. 2D nonlinear reaction-diffusion equation
  8. Conclusion
  9. Appendix
  10. Fourier features networks
  11. GB PINN Algorithm Motivation
  12. Numerical Experiments Setup
  13. Results of 1D singular perturbation
  14. Results of 2D singular perturbation with boundary layers
  15. Results of 2D singular perturbation with an interior boundary layer
  16. ...and 3 more sections

Figures (7)

  • Figure 1: Prediction of singular perturbation problem by GB PINNs, $\varepsilon=10^{-4}$. Left: predicted solution (black) v.s. ground truth (red). Right: pointwise error.
  • Figure 2: Prediction of singular perturbation problem by PINNs for ablation study, $\varepsilon=10^{-4}$. Left: predicted solution (black) v.s. ground truth (red). Right: pointwise error.
  • Figure 3: Prediction of 2D singular perturbation with boundary problem by GB PINNs, $\varepsilon=10^{-3}$. Left: predicted solution. Middle: ground truth. Right: pointwise error.
  • Figure 4: Prediction of 2D singular perturbation with boundary problem by PINNs, $\varepsilon=10^{-3}$. Left: predicted solution. Middle: ground truth. Right: pointwise error.
  • Figure 5: Prediction of 2D singular perturbation with interior boundary problem by GB PINNs, $\varepsilon=10^{-4}$. Left: predicted solution. Middle: ground truth. Right: pointwise error.
  • ...and 2 more figures