Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Coordinate Encoding on Linear Grids for Physics-Informed Neural Networks

Tetsuro Tsuchino, Motoki Shiga

Abstract

In solving partial differential equations (PDEs), machine learning utilizing physical laws has received considerable attention owing to advantages such as mesh-free solutions, unsupervised learning, and feasibility for solving high-dimensional problems. An effective approach is based on physics-informed neural networks (PINNs), which are based on deep neural networks known for their excellent performance in various academic and industrial applications. However, PINNs struggled with model training owing to significantly slow convergence because of a spectral bias problem. In this study, we propose a PINN-based method equipped with a coordinate-encoding layer on linear grid cells. The proposed method improves the training convergence speed by separating the local domains using grid cells. Moreover, it reduces the overall computational cost by using axis-independent linear grid cells. The method also achieves efficient and stable model training by adequately interpolating the encoded coordinates between grid points using natural cubic splines, which guarantees continuous derivative functions of the model computed for the loss functions. The results of numerical experiments demonstrate the effective performance and efficient training convergence speed of the proposed method.

Coordinate Encoding on Linear Grids for Physics-Informed Neural Networks

Abstract

In solving partial differential equations (PDEs), machine learning utilizing physical laws has received considerable attention owing to advantages such as mesh-free solutions, unsupervised learning, and feasibility for solving high-dimensional problems. An effective approach is based on physics-informed neural networks (PINNs), which are based on deep neural networks known for their excellent performance in various academic and industrial applications. However, PINNs struggled with model training owing to significantly slow convergence because of a spectral bias problem. In this study, we propose a PINN-based method equipped with a coordinate-encoding layer on linear grid cells. The proposed method improves the training convergence speed by separating the local domains using grid cells. Moreover, it reduces the overall computational cost by using axis-independent linear grid cells. The method also achieves efficient and stable model training by adequately interpolating the encoded coordinates between grid points using natural cubic splines, which guarantees continuous derivative functions of the model computed for the loss functions. The results of numerical experiments demonstrate the effective performance and efficient training convergence speed of the proposed method.
Paper Structure (24 sections, 23 equations, 10 figures, 9 tables)

This paper contains 24 sections, 23 equations, 10 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Physics-Informed Neural Network (PINN)
  4. Encoding Coordinates into Feature Vectors
  5. Tensor Factorization Models
  6. Our Proposed Method: Coordinate Encoding on Linear Grid (CELG)
  7. Notations and Problem Setting
  8. The Proposed Method
  9. Coordinate Encoding
  10. Integrating Encoded Features
  11. Computational Complexity of Coordinate Encoding
  12. Numerical Experiments
  13. PDEs
  14. Multi-Band Poisson Equation
  15. Burgers Equation
  16. ...and 9 more sections

Figures (10)

  • Figure 1: Schematic of the proposed method, coordinate encoding on linear grid (CELG).
  • Figure 2: Training results utilizing linear, cosine with/without multigrid (MG), Hermite and natural interpolations for 1D Poisson PDE. (a) Predictions of $u(x)$, (b) first and (c) second derivatives. (d) Learning curves. In (a)--(c), the analytical solution is shown as a solid gray line.
  • Figure 3: Analytical solutions of (a) 1D and (b) 2D multi-band Poisson PDEs.
  • Figure 4: Prediction results for the 1D multi-band Poisson PDE during training at (a) 100, (b) 1,000, (c) 5,000, and (d) 10,000 training epochs. The grid parameter was set to $R = 16$ in PIXEL, H-Spline, and CELG.
  • Figure 5: Learning curves for the 1D multi-band Poisson PDE. The grid parameters were set to (a) $R=4$, (b) 8, (c) 16, and (d) 32.
  • ...and 5 more figures