Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CP-PINNs: Data-Driven Changepoints Detection in PDEs Using Online Optimized Physics-Informed Neural Networks

Zhikang Dong, Pawel Polak

TL;DR

The paper addresses inverse PDE problems with abrupt time-varying dynamics by introducing CP-PINNs, which couple Physics-Informed Neural Networks with a Total Variation penalty on the time-varying parameter $\lambda(t)$ to detect changepoints and estimate parameters. It also presents an online loss-weighting scheme based on exponentiated-descent updates, with a theoretical regret bound that guarantees stability of the learning process. Empirically, CP-PINNs outperform standard PINNs in scenarios with changepoints (e.g., advection-diffusion and Navier–Stokes with viscosity changes), offering accurate changepoint detection and improved PDE parameter estimates while recovering PINNs when no changepoints are present. This approach enables robust, data-driven PDE discovery under abrupt dynamical changes and supports real-time changepoint detection in engineering applications.

Abstract

We investigate the inverse problem for Partial Differential Equations (PDEs) in scenarios where the parameters of the given PDE dynamics may exhibit changepoints at random time. We employ Physics-Informed Neural Networks (PINNs) - universal approximators capable of estimating the solution of any physical law described by a system of PDEs, which serves as a regularization during neural network training, restricting the space of admissible solutions and enhancing function approximation accuracy. We demonstrate that when the system exhibits sudden changes in the PDE dynamics, this regularization is either insufficient to accurately estimate the true dynamics, or it may result in model miscalibration and failure. Consequently, we propose a PINNs extension using a Total-Variation penalty, which allows to accommodate multiple changepoints in the PDE dynamics and significantly improves function approximation. These changepoints can occur at random locations over time and are estimated concurrently with the solutions. Additionally, we introduce an online learning method for re-weighting loss function terms dynamically. Through empirical analysis using examples of various equations with parameter changes, we showcase the advantages of our proposed model. In the absence of changepoints, the model reverts to the original PINNs model. However, when changepoints are present, our approach yields superior parameter estimation, improved model fitting, and reduced training error compared to the original PINNs model.

CP-PINNs: Data-Driven Changepoints Detection in PDEs Using Online Optimized Physics-Informed Neural Networks

TL;DR

The paper addresses inverse PDE problems with abrupt time-varying dynamics by introducing CP-PINNs, which couple Physics-Informed Neural Networks with a Total Variation penalty on the time-varying parameter to detect changepoints and estimate parameters. It also presents an online loss-weighting scheme based on exponentiated-descent updates, with a theoretical regret bound that guarantees stability of the learning process. Empirically, CP-PINNs outperform standard PINNs in scenarios with changepoints (e.g., advection-diffusion and Navier–Stokes with viscosity changes), offering accurate changepoint detection and improved PDE parameter estimates while recovering PINNs when no changepoints are present. This approach enables robust, data-driven PDE discovery under abrupt dynamical changes and supports real-time changepoint detection in engineering applications.

Abstract

We investigate the inverse problem for Partial Differential Equations (PDEs) in scenarios where the parameters of the given PDE dynamics may exhibit changepoints at random time. We employ Physics-Informed Neural Networks (PINNs) - universal approximators capable of estimating the solution of any physical law described by a system of PDEs, which serves as a regularization during neural network training, restricting the space of admissible solutions and enhancing function approximation accuracy. We demonstrate that when the system exhibits sudden changes in the PDE dynamics, this regularization is either insufficient to accurately estimate the true dynamics, or it may result in model miscalibration and failure. Consequently, we propose a PINNs extension using a Total-Variation penalty, which allows to accommodate multiple changepoints in the PDE dynamics and significantly improves function approximation. These changepoints can occur at random locations over time and are estimated concurrently with the solutions. Additionally, we introduce an online learning method for re-weighting loss function terms dynamically. Through empirical analysis using examples of various equations with parameter changes, we showcase the advantages of our proposed model. In the absence of changepoints, the model reverts to the original PINNs model. However, when changepoints are present, our approach yields superior parameter estimation, improved model fitting, and reduced training error compared to the original PINNs model.
Paper Structure (7 sections, 2 theorems, 25 equations, 5 figures, 1 algorithm)

This paper contains 7 sections, 2 theorems, 25 equations, 5 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Model and Estimation
  3. Physics-Informed Neural Network
  4. Total Variation Regularization
  5. Exponentiated Descent Weights Update Method for Loss Function
  6. Empirical Results
  7. Conclusions

Key Result

Lemma 2.2

The solution to eq:objective_func is given by where and $\gamma$ is such that the vector $\mathbf{w}^{(k)}$ is guaranteed to remain in the simplex $\mathcal{S}_3$, i.e.,

Figures (5)

  • Figure 1: The dataset is partitioned based on spatial information, with each batch encompassing the full temporal information. In the online learning approach, the network is trained using the previous distribution of loss weights and updated based on the data from the subsequent batch.
  • Figure 2: (Left) Parameter estimation of CP-PINNs and PINNs. (Right) Loss weights distribution across time.
  • Figure 3: The solution of 1D advection-equation with two breakpoints.
  • Figure 4: (Left) Parameter estimation of CP-PINNs and PINNs. (Right) Loss weights distribution across time.
  • Figure 5: The stream-wise and transverse velocity of 2D Navier-Stokes equation with two breakpoints.

Theorems & Definitions (6)

  • Definition 2.1
  • Lemma 2.2
  • Corollary 2.3
  • Remark 2.4
  • proof
  • proof