Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Stabilized Adaptive Loss and Residual-Based Collocation for Physics-Informed Neural Networks

Divyavardhan Singh, Shubham Kamble, Dimple Sonone, Kishor Upla

TL;DR

A new adaptive loss balancing scheme using smoothed gradient norms to ensure satisfaction of initial and boundary conditions and an adaptive residual-based collocation scheme to improve the accuracy of solutions in the regions with high physics residuals are developed.

Abstract

Physics-Informed Neural Networks (PINNs) have been recognized as a mesh-free alternative to solve partial differential equations where physics information is incorporated. However, in dealing with problems characterized by high stiffness or shock-dominated dynamics, traditional PINNs have been found to have limitations, including unbalanced training and inaccuracy in solution, even with small physics residuals. In this research, we seek to address these limitations using the viscous Burgers' equation with low viscosity and the Allen-Cahn equation as test problems. In addressing unbalanced training, we have developed a new adaptive loss balancing scheme using smoothed gradient norms to ensure satisfaction of initial and boundary conditions. Further, to address inaccuracy in the solution, we have developed an adaptive residual-based collocation scheme to improve the accuracy of solutions in the regions with high physics residuals. The proposed new approach significantly improves solution accuracy with consistent satisfaction of physics residuals. For instance, in the case of Burgers' equation, the relative L2 error is reduced by about 44 percent compared to traditional PINNs, while for the Allen-Cahn equation, the relative L2 error is reduced by approximately 70 percent. Additionally, we show the trustworthy solution comparison of the proposed method using a robust finite difference solver.

Stabilized Adaptive Loss and Residual-Based Collocation for Physics-Informed Neural Networks

TL;DR

A new adaptive loss balancing scheme using smoothed gradient norms to ensure satisfaction of initial and boundary conditions and an adaptive residual-based collocation scheme to improve the accuracy of solutions in the regions with high physics residuals are developed.

Abstract

Physics-Informed Neural Networks (PINNs) have been recognized as a mesh-free alternative to solve partial differential equations where physics information is incorporated. However, in dealing with problems characterized by high stiffness or shock-dominated dynamics, traditional PINNs have been found to have limitations, including unbalanced training and inaccuracy in solution, even with small physics residuals. In this research, we seek to address these limitations using the viscous Burgers' equation with low viscosity and the Allen-Cahn equation as test problems. In addressing unbalanced training, we have developed a new adaptive loss balancing scheme using smoothed gradient norms to ensure satisfaction of initial and boundary conditions. Further, to address inaccuracy in the solution, we have developed an adaptive residual-based collocation scheme to improve the accuracy of solutions in the regions with high physics residuals. The proposed new approach significantly improves solution accuracy with consistent satisfaction of physics residuals. For instance, in the case of Burgers' equation, the relative L2 error is reduced by about 44 percent compared to traditional PINNs, while for the Allen-Cahn equation, the relative L2 error is reduced by approximately 70 percent. Additionally, we show the trustworthy solution comparison of the proposed method using a robust finite difference solver.
Paper Structure (31 sections, 20 equations, 2 figures, 4 tables)

This paper contains 31 sections, 20 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Adaptive Loss Weighting and Optimization in PINNs
  4. Adaptive Sampling and Collocation Strategies
  5. Problem Formulation
  6. Governing Equations
  7. Viscous Burgers' Equation
  8. Allen--Cahn Equation
  9. Physics-Informed Neural Network Approximation
  10. Loss Function Components
  11. PDE Residual Loss
  12. Initial Condition Loss
  13. Boundary Condition Loss
  14. Total Training Objective
  15. Evaluation Metrics
  16. ...and 16 more sections

Figures (2)

  • Figure 1: Mean boundary condition error for Burgers' and Allen--Cahn equations. The proposed stabilized adaptive loss formulation significantly improves boundary enforcement compared to the standard PINN raissi2019wang2021jagtap2020.
  • Figure 2: Mean PDE residual comparison between standard and adaptive PINNs raissi2019xiang2022. Adaptive formulations preserve strong physics satisfaction while improving training stability.