Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fine-Tuning DeepONets to Enhance Physics-informed Neural Networks for solving Partial Differential Equations

Sidi Wu

TL;DR

A parameter-efficient approach that fine-tunes pre-trained DeepONet models within the PINN framework (FTO-PINN), enabling more efficient meshless PDE solving and significantly reduces the training time of vanilla PINNs while maintaining comparable accuracy.

Abstract

Physics-Informed Neural Networks (PINNs) have emerged as powerful tools for solving partial differential equations (PDEs). However, training PINNs from scratch is often computationally intensive and time-consuming. To address this problem, we propose a parameter-efficient approach that fine-tunes pre-trained DeepONet models within the PINN framework (FTO-PINN), enabling more efficient meshless PDE solving. Specifically, we freeze the weights of the pre-trained DeepONet model and fine-tune the output of the branch net by incorporating a small number of new trainable parameters, which can be quickly determined using least-squares techniques. Additionally, we introduce trunk net expansions and low-rank adaptation strategies to further enhance the performance of FTO-PINN. The effectiveness of our proposed method is demonstrated through a series of numerical experiments across various types of PDEs. FTO-PINN significantly reduces the training time of vanilla PINNs while maintaining comparable accuracy, and outperforms DeepONet, which is pre-trained on general function data, in both fidelity and generalization capabilities.

Fine-Tuning DeepONets to Enhance Physics-informed Neural Networks for solving Partial Differential Equations

TL;DR

A parameter-efficient approach that fine-tunes pre-trained DeepONet models within the PINN framework (FTO-PINN), enabling more efficient meshless PDE solving and significantly reduces the training time of vanilla PINNs while maintaining comparable accuracy.

Abstract

Physics-Informed Neural Networks (PINNs) have emerged as powerful tools for solving partial differential equations (PDEs). However, training PINNs from scratch is often computationally intensive and time-consuming. To address this problem, we propose a parameter-efficient approach that fine-tunes pre-trained DeepONet models within the PINN framework (FTO-PINN), enabling more efficient meshless PDE solving. Specifically, we freeze the weights of the pre-trained DeepONet model and fine-tune the output of the branch net by incorporating a small number of new trainable parameters, which can be quickly determined using least-squares techniques. Additionally, we introduce trunk net expansions and low-rank adaptation strategies to further enhance the performance of FTO-PINN. The effectiveness of our proposed method is demonstrated through a series of numerical experiments across various types of PDEs. FTO-PINN significantly reduces the training time of vanilla PINNs while maintaining comparable accuracy, and outperforms DeepONet, which is pre-trained on general function data, in both fidelity and generalization capabilities.

Paper Structure

This paper contains 22 sections, 2 theorems, 45 equations, 8 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related works
  3. Preliminary
  4. Feedforward neural network
  5. Physics-informed neural networks
  6. Deep operator networks
  7. Methodology
  8. Fine-tuning with a pre-trained DeepONet
  9. Linear PDEs
  10. Nonlinear PDEs
  11. Interface problems
  12. Expansion of trunk nets
  13. Further fine-tune the trunk net as needed.
  14. Connection with other numerical methods
  15. Numerical Results
  16. ...and 7 more sections

Key Result

Proposition 3.1

Suppose that $X$ and $Y$ are Banach spaces, $K \subset X$ are compact set, $\mathcal{G}: K\rightarrow \mathcal{G}(K)\subset Y$ is a continuous operator. For $\varepsilon>0$, assume there exists a DeepONet $\mathcal{G}_{\boldsymbol{\theta}}$ of form (eq: DeepONet) such that If the function space $\mathcal{F} \subset \mathcal{G}(K)$, then where $\langle \cdot, \cdot \rangle$ denotes the dot produc

Figures (8)

  • Figure 1: Schematic diagram of FTO-PINN learning procedure.
  • Figure 2: Advection equation: Variation of the $L^\infty$ and relative $L^2$ errors of FTO-PINN and RWM with respect to DOF.
  • Figure 3: Advection equation: (First column) Coefficient function $a(x)$ generated with different length scales. (Second to sixth columns) The corresponding reference solutions and point-wise errors for PI-DeepONet, PINN, FTO-PINN (700) and RWM (700).
  • Figure 4: Diffusion-reaction equation: The $L^\infty$ and relative $L^2$ errors of FT0-PINN and RWM over 100 randomly sampled $f(x)$. Here, red dashed lines indicate the mean error of pre-trained DeepONet.
  • Figure 5: Diffusion-reaction equation: (First column) Source terms $f(x)$ generated with different length scales. (Second to sixth columns) The reference solutions $u(\mathbf{x})$ and point-wise errors for PI-DeepONet, PINN, FTO-PINN (500) and RWM (500).
  • ...and 3 more figures

Theorems & Definitions (4)

  • Remark 3.1
  • Remark 3.2
  • Proposition 3.1
  • Proposition 3.2