Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SVD-PINNs: Transfer Learning of Physics-Informed Neural Networks via Singular Value Decomposition

Yihang Gao, Ka Chun Cheung, Michael K. Ng

TL;DR

Numerical experiments show that SVD-PINNs work for solving a class of PDEs with different but close right-hand-side functions, and transfer learning methods may reduce the cost for PINNs in solving aclass of P DEs.

Abstract

Physics-informed neural networks (PINNs) have attracted significant attention for solving partial differential equations (PDEs) in recent years because they alleviate the curse of dimensionality that appears in traditional methods. However, the most disadvantage of PINNs is that one neural network corresponds to one PDE. In practice, we usually need to solve a class of PDEs, not just one. With the explosive growth of deep learning, many useful techniques in general deep learning tasks are also suitable for PINNs. Transfer learning methods may reduce the cost for PINNs in solving a class of PDEs. In this paper, we proposed a transfer learning method of PINNs via keeping singular vectors and optimizing singular values (namely SVD-PINNs). Numerical experiments on high dimensional PDEs (10-d linear parabolic equations and 10-d Allen-Cahn equations) show that SVD-PINNs work for solving a class of PDEs with different but close right-hand-side functions.

SVD-PINNs: Transfer Learning of Physics-Informed Neural Networks via Singular Value Decomposition

TL;DR

Numerical experiments show that SVD-PINNs work for solving a class of PDEs with different but close right-hand-side functions, and transfer learning methods may reduce the cost for PINNs in solving aclass of P DEs.

Abstract

Physics-informed neural networks (PINNs) have attracted significant attention for solving partial differential equations (PDEs) in recent years because they alleviate the curse of dimensionality that appears in traditional methods. However, the most disadvantage of PINNs is that one neural network corresponds to one PDE. In practice, we usually need to solve a class of PDEs, not just one. With the explosive growth of deep learning, many useful techniques in general deep learning tasks are also suitable for PINNs. Transfer learning methods may reduce the cost for PINNs in solving a class of PDEs. In this paper, we proposed a transfer learning method of PINNs via keeping singular vectors and optimizing singular values (namely SVD-PINNs). Numerical experiments on high dimensional PDEs (10-d linear parabolic equations and 10-d Allen-Cahn equations) show that SVD-PINNs work for solving a class of PDEs with different but close right-hand-side functions.
Paper Structure (16 sections, 13 equations, 7 figures, 1 algorithm)

This paper contains 16 sections, 13 equations, 7 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Transfer Learning
  3. The Contribution
  4. Some Potential Limitations
  5. Notations
  6. Methods
  7. Physics-Informed Neural Networks (PINNs)
  8. Transfer Learning of PINNs
  9. SVD-PINNs
  10. Motivation
  11. Algorithm
  12. Advantages
  13. Experimental Results
  14. Linear Parabolic Equations
  15. Allen-Cahn Equations
  16. ...and 1 more sections

Figures (7)

  • Figure 1: Trajectories of the relative error for the SVD-PINNs with different optimizers and learning rates in solving the $10$-dimensional linear parabolic equation ($\epsilon = 0.5$).
  • Figure 2: Trajectories of the relative error for the SVD-PINNs with different optimizers and learning rates in solving the $10$-dimensional linear parabolic equation ($\epsilon = 2$).
  • Figure 3: Trajectories of the relative error for the SVD-PINNs with different optimizers and learning rates in solving the $10$-dimensional Allen-Cahn equation ($\epsilon = 0.5$).
  • Figure 4: Trajectories of the relative error for the SVD-PINNs with different optimizers and learning rates in solving the $10$-dimensional Allen-Cahn equation ($\epsilon = 2$).
  • Figure 5: Comparisons of SVD-PINNs with the full training and the transfer learning with a frozen hidden layer in solving the $10$-dimensional Allen-Cahn equation ($\epsilon = 50$).
  • ...and 2 more figures