Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Gradient Enhanced Self-Training Physics-Informed Neural Network (gST-PINN) for Solving Nonlinear Partial Differential Equations

Narayan S Iyer, Bivas Bhaumik, Ram S Iyer, Satyasaran Changdar

TL;DR

The paper tackles the challenge of solving nonlinear PDEs with limited labeled data by introducing gST-PINN, a gradient-enhanced self-training Physics-Informed Neural Network. The method combines gradient-based residual information from gPINN with a gradient-enhanced pseudo-labeling strategy to generate supervised data from unlabeled collocation points, improving convergence and accuracy. Empirical results on Burgers, diffusion-reaction, and diffusion-sorption equations show that gST-PINN outperforms standard PINN and other variants, achieving MSEs as low as $\mathcal{O}(10^{-5})$ in several cases and maintaining robustness even without labeled data. This approach offers a semi-supervised, physics-guided solver that enhances precision and generalization in multi-physics PDE contexts where labeled data are scarce, with potential extensions to complex geometries, high-dimensional domains, and fractional differential equations.

Abstract

Partial differential equations (PDEs) provide a mathematical foundation for simulating and understanding intricate behaviors in both physical sciences and engineering. With the growing capabilities of deep learning, data$-$driven approaches like Physics$-$Informed Neural Networks (PINNs) have been developed, offering a mesh$-$free, analytic type framework for efficiently solving PDEs across a wide range of applications. However, traditional PINNs often struggle with challenges such as limited precision, slow training dynamics, lack of labeled data availability, and inadequate handling of multi$-$physics interactions. To overcome these challenging issues of PINNs, we proposed a Gradient Enhanced Self$-$Training PINN (gST$-$PINN) method that specifically introduces a gradient based pseudo point self$-$learning algorithm for solving PDEs. We tested the proposed method on three different types of PDE problems from various fields, each representing distinct scenarios. The effectiveness of the proposed method is evident, as the PINN approach for solving the Burgers$'$ equation attains a mean square error (MSE) on the order of $10^{-3}$, while the diffusion$-$sorption equation achieves an MSE on the order of $10^{-4}$ after 12,500 iterations, with no further improvement as the iterations increase. In contrast, the MSE for both PDEs in the gST$-$PINN model continues to decrease, demonstrating better generalization and reaching an MSE on the order of $10^{-5}$ after 18,500 iterations. Furthermore, the results show that the proposed purely semi$-$supervised gST$-$PINN consistently outperforms the standard PINN method in all cases, even when solution of the PDEs are unavailable. It generalizes both PINN and Gradient$-$enhanced PINN (gPINN), and can be effectively applied in scenarios prone to low accuracy and convergence issues, particularly in the absence of labeled data.

Gradient Enhanced Self-Training Physics-Informed Neural Network (gST-PINN) for Solving Nonlinear Partial Differential Equations

TL;DR

The paper tackles the challenge of solving nonlinear PDEs with limited labeled data by introducing gST-PINN, a gradient-enhanced self-training Physics-Informed Neural Network. The method combines gradient-based residual information from gPINN with a gradient-enhanced pseudo-labeling strategy to generate supervised data from unlabeled collocation points, improving convergence and accuracy. Empirical results on Burgers, diffusion-reaction, and diffusion-sorption equations show that gST-PINN outperforms standard PINN and other variants, achieving MSEs as low as in several cases and maintaining robustness even without labeled data. This approach offers a semi-supervised, physics-guided solver that enhances precision and generalization in multi-physics PDE contexts where labeled data are scarce, with potential extensions to complex geometries, high-dimensional domains, and fractional differential equations.

Abstract

Partial differential equations (PDEs) provide a mathematical foundation for simulating and understanding intricate behaviors in both physical sciences and engineering. With the growing capabilities of deep learning, datadriven approaches like PhysicsInformed Neural Networks (PINNs) have been developed, offering a meshfree, analytic type framework for efficiently solving PDEs across a wide range of applications. However, traditional PINNs often struggle with challenges such as limited precision, slow training dynamics, lack of labeled data availability, and inadequate handling of multiphysics interactions. To overcome these challenging issues of PINNs, we proposed a Gradient Enhanced SelfTraining PINN (gSTPINN) method that specifically introduces a gradient based pseudo point selflearning algorithm for solving PDEs. We tested the proposed method on three different types of PDE problems from various fields, each representing distinct scenarios. The effectiveness of the proposed method is evident, as the PINN approach for solving the Burgers equation attains a mean square error (MSE) on the order of , while the diffusionsorption equation achieves an MSE on the order of after 12,500 iterations, with no further improvement as the iterations increase. In contrast, the MSE for both PDEs in the gSTPINN model continues to decrease, demonstrating better generalization and reaching an MSE on the order of after 18,500 iterations. Furthermore, the results show that the proposed purely semisupervised gSTPINN consistently outperforms the standard PINN method in all cases, even when solution of the PDEs are unavailable. It generalizes both PINN and Gradientenhanced PINN (gPINN), and can be effectively applied in scenarios prone to low accuracy and convergence issues, particularly in the absence of labeled data.

Paper Structure

This paper contains 17 sections, 28 equations, 9 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Scientific And Physics Informed Machine Learning For Solving Non Linear PDEs
  3. Self Training
  4. Related works
  5. Overview of the paper and our contribution
  6. Methodology
  7. Physics Informed Neural Network (PINN) for solving PDEs
  8. Gradient enhanced physics informed neural network (gPINN) for solving PDEs
  9. Gradient Enhanced Self Training Physics Informed Neural Network (gST-PINN): Proposed Approach
  10. Overview of the framework
  11. Pseudo Point Generation Strategy
  12. Experimental Results
  13. Burgers Equation
  14. Diffusion-Reaction Equation
  15. Diffusion-Sorption Equation
  16. ...and 2 more sections

Figures (9)

  • Figure 1: Pipeline/Architecture of the gST-PINN model (section (\ref{['sec_3_gSTPINN']})). The green line depicts the neural network training process fulfilled by minimizing the loss function given by equation (\ref{['loss_with_grad']}) (section (\ref{['sec_2.1_PINN_for_PDES']})) and the red line represents the gradient enhanced pseudo point generation algorithm/strategy accomplished by filtering and labeling the collocation points with considerably small PDE and gradient residuals given by equations (\ref{['PDE-residual']}) and (\ref{['grad_residual']}) (section(\ref{['sec_3.2_pse_gen_stra']}), Algorithm \ref{['algorithm_1']}. Both the processes, apart from the incorporation of the physics based information (PDE residual, initial and boundary conditions, additional constraints and symmetries, etc.), also utilizes the gradient details (section (\ref{['sec_2.2-gPINN_for_PDE']})) for escalating the overall accuracy of the model.
  • Figure 2: Functional form of initial and boundary conditions of (a): Burgers' equation (section(\ref{["sec_4.1_burgers'"]})), (b): diffusion reaction equation (section (\ref{['sec_4.2_diff_react']})), (c): diffusion sorption equation (section(\ref{['sec_4.3_diff_sorp']})).
  • Figure 3: Plots of $MSE$ error in $\hat{u}$ vs number of iterations in gST-PINN and PINN models for the approximation of solution to the Burgers' equation (\ref{["sec_4.1_burgers'"]}) and diffusion-sorption equation (\ref{['sec_4.3_diff_sorp']}). It is conspicuous that the $MSE$ errors in PINN model converge very quickly (for example, approximately after 7500 iterations in case of Burgers' equation, eventually achieving the $\mathcal{O}(10^{-3})$ and 12500 iterations in case of diffusion-sorption equation, eventually achieving $\mathcal{O}(10^{-4})$), while the $MSE$ errors in gST-PINN model continue to decrease till the last iteration.
  • Figure 4: (Section (\ref{["sec_4.1_burgers'"]}), Burgers' equation)(a): Comparison of the actual solution $u$ of Burgers' equation with its predicted solutions $\hat{u}$ using gST-PINN and PINN models. (b): Comparison of the absolute error in $u$, i.e., $|u-\hat{u}|$ predicted by the gST-PINN and PINN model. From both (a) and (b), it can be clearly observed that the prediction $\hat{u}$ of the solution $u$ for Burgers' equation performed by the gST-PINN model is more accurate in comparison with the PINN model, i.e., $|u(x)-\hat{u}(x)|_{gST-PINN} \leq |u(x)-\hat{u}(x)|_{PINN}$, $~~\forall x \in \Omega_{Burgers'}$, where $\Omega_{Burgers'}$ denotes the domain on which the Burgers' equation is defined.
  • Figure 5: (Section (\ref{["sec_4.1_burgers'"]}), Burgers' equation) Performance comparison analysis of gST-PINN model and the normal PINN model for solving the Burgers' equation. The analysis has been done by studying the "$\textbf{u}$ vs $\textbf{x}$" plots at three different time instances, $\textbf{t=0.3,~0.6,~0.9}$. It has been examined from the plots that, initially (t=0.3), both the models exhibit roughly identical accuracy, but at later intervals (t=0.6, 0.9), the accuracy of the PINN model considerably deteriorates whereas the gST-PINN model succeeds in preserving the accuracy level exhibited initially, thereby proving to be more consistent compared to the PINN model.
  • ...and 4 more figures