Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Physics-informed machine learning for reconstruction of dynamical systems with invariant measure score matching

Yongsheng Chen, Suddhasattwa Das, Wei Guo, Xinghui Zhong

TL;DR

This work addresses reconstructing dynamical systems from unlabeled point-cloud data by leveraging the invariant measure of the system via the stationary Fokker-Planck equation. It introduces PINN-IMSM, a mesh-free, two-stage framework that first learns the score of the invariant measure through multi-scale denoising score matching and then embeds this score in a PINN to recover the drift field under a score-based FP constraint. To overcome ill-posedness in high dimensions, the problem is recast as a PDE-constrained optimization that seeks the minimal-energy velocity field, solved efficiently with a stochastic augmented Lagrangian method, with a proven local Lipschitz dependence of the velocity on the score. The method demonstrates accurate reconstruction of invariant measures and dynamic behavior for systems up to five dimensions, including chaotic Lorenz models, and shows improved PDE residuals over standard PINN approaches. Overall, PINN-IMSM provides a scalable, mesh-free approach to learning high-dimensional dynamical laws from unlabeled data, with strong theoretical and empirical support for stability and uniqueness.

Abstract

In this paper, we develop a novel mesh-free framework, termed physics-informed neural networks with invariant measure score matching (PINN-IMSM), for reconstructing dynamical systems from unlabeled point-cloud data that capture the system's invariant measure. The invariant density satisfies the steady-state Fokker-Planck (FP) equation. We reformulate this equation in terms of its score function (the gradient of the log-density), which is estimated directly from data via denoising score matching, thereby bypassing explicit density estimation. This learned score is then embedded into a physics-informed neural network (PINN) to reconstruct the drift velocity field under the resulting score-based FP equation. The mesh-free nature of PINNs allows the framework to scale to higher dimensions, avoiding the curse of dimensionality inherent in mesh-based methods. To address the ill-posedness of high-dimensional inverse problems, we recast the problem as a PDE-constrained optimization that seeks the minimal-energy velocity field. Under suitable conditions, we prove that this problem admits a unique solution that depends continuously on the score function. The constrained formulation is solved using a stochastic augmented Lagrangian method. Numerical experiments on representative dynamical systems, including the Van der Pol oscillator, an active swimmer in an anharmonic trap, and the chaotic Lorenz-63 and Lorenz-96 systems, demonstrate that PINN-IMSM accurately recovers invariant measures and reconstructs faithful dynamical behavior for problems in up to five dimensions.

Physics-informed machine learning for reconstruction of dynamical systems with invariant measure score matching

TL;DR

This work addresses reconstructing dynamical systems from unlabeled point-cloud data by leveraging the invariant measure of the system via the stationary Fokker-Planck equation. It introduces PINN-IMSM, a mesh-free, two-stage framework that first learns the score of the invariant measure through multi-scale denoising score matching and then embeds this score in a PINN to recover the drift field under a score-based FP constraint. To overcome ill-posedness in high dimensions, the problem is recast as a PDE-constrained optimization that seeks the minimal-energy velocity field, solved efficiently with a stochastic augmented Lagrangian method, with a proven local Lipschitz dependence of the velocity on the score. The method demonstrates accurate reconstruction of invariant measures and dynamic behavior for systems up to five dimensions, including chaotic Lorenz models, and shows improved PDE residuals over standard PINN approaches. Overall, PINN-IMSM provides a scalable, mesh-free approach to learning high-dimensional dynamical laws from unlabeled data, with strong theoretical and empirical support for stability and uniqueness.

Abstract

In this paper, we develop a novel mesh-free framework, termed physics-informed neural networks with invariant measure score matching (PINN-IMSM), for reconstructing dynamical systems from unlabeled point-cloud data that capture the system's invariant measure. The invariant density satisfies the steady-state Fokker-Planck (FP) equation. We reformulate this equation in terms of its score function (the gradient of the log-density), which is estimated directly from data via denoising score matching, thereby bypassing explicit density estimation. This learned score is then embedded into a physics-informed neural network (PINN) to reconstruct the drift velocity field under the resulting score-based FP equation. The mesh-free nature of PINNs allows the framework to scale to higher dimensions, avoiding the curse of dimensionality inherent in mesh-based methods. To address the ill-posedness of high-dimensional inverse problems, we recast the problem as a PDE-constrained optimization that seeks the minimal-energy velocity field. Under suitable conditions, we prove that this problem admits a unique solution that depends continuously on the score function. The constrained formulation is solved using a stochastic augmented Lagrangian method. Numerical experiments on representative dynamical systems, including the Van der Pol oscillator, an active swimmer in an anharmonic trap, and the chaotic Lorenz-63 and Lorenz-96 systems, demonstrate that PINN-IMSM accurately recovers invariant measures and reconstructs faithful dynamical behavior for problems in up to five dimensions.
Paper Structure (22 sections, 6 theorems, 81 equations, 7 figures, 1 algorithm)

This paper contains 22 sections, 6 theorems, 81 equations, 7 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Stochastic differential equations, invariant measures, and Fokker-Planck equations
  4. Score estimation
  5. Physics-informed neural networks
  6. PINN-IMSM: PINN with invariant measure score matching
  7. Problem reformulation and method overview
  8. Score matching for invariant measures
  9. PINN-based velocity field reconstruction
  10. The ill-posedness challenge in the standard PINN approach
  11. Ensuring well-posedness: PDE-constrained optimization approach
  12. Numerical implementation: stochastic augmented Lagrangian method
  13. Inner-loop optimization via stochastic gradient descent
  14. Outer-loop updating of multipliers and penalties
  15. Summary of the PINN-IMSM framework
  16. ...and 7 more sections

Key Result

Theorem 3.1

Assume ${\bm{s}}\in W^{1,\infty}(\Omega;\mathbb{R}^d)$ and that $q_{{\bm{s}}}=|{\bm{s}}|^2+\nabla\cdot{\bm{s}}\ge0$ almost everywhere in $\Omega$. Then the constrained optimization problem eq:true-prob admits a unique minimizer ${\bm{v}}^*$. Moreover, the solution map ${\bm{s}}\mapsto {\bm{v}}^*$ is

Figures (7)

  • Figure 3.1: Velocity field reconstruction from trajectory data without explicit time labels. From trajectory measurements (left), we first reconstruct the score function via denoising score matching (middle). This reconstructed score is then integrated into the PINN framework to infer the velocity field across the state space (right).
  • Figure 3.2: Schematic outline of the PINN-IMSM workflow. The goal is to reconstruct the drift term of an SDE from unlabeled trajectory data by combining invariant measure score matching with PINNs. Starting from an SDE with constant isotropic diffusion, the framework transforms the problem into a steady-state FP equation, which is then reformulated using the score function of the invariant density. The method proceeds in two stages: first, estimating the score function via multi-scale denoising score matching; second, reconstructing the velocity field by solving a well-posed PDE-constrained optimization problem within the stochastic augmented Lagrangian framework.
  • Figure 4.1: Van der Pol oscillator \ref{['eq:van']} with $D=0.05$. Top: ground-truth velocity field (left), reference invariant density (middle), and noisy trajectory samples (right). Bottom: PINN-IMSM reconstructed velocity field ${\bm{v}}_{\theta_2^*}$ (left), learned invariant density (middle), and samples generated by Langevin dynamics from the learned score ${\bm{s}}_{\theta_1^*}$ (right).
  • Figure 4.2: PDE residual during training for the standard PINN and PINN-IMSM ("Aug_lag") on the Van der Pol oscillator.
  • Figure 4.3: Invariant density estimates for the active swimmer model \ref{['eq:swim']} with $\gamma=0.1$ and $D=1.0$. Left: learned invariant density from the reconstructed velocity. Right: reference invariant density from the ground-truth velocity.
  • ...and 2 more figures

Theorems & Definitions (16)

  • Theorem 3.1: Well-posedness of \ref{['eq:true-prob']}
  • Remark 3.1
  • Example 4.1: Van der Pol oscillator
  • Example 4.2: Active swimmer in an anharmonic trap
  • Example 4.3: Lorenz-63 system
  • Example 4.4: Lorenz-96 system
  • Theorem A.1: Implicit Function Theorem krantz2002implicit
  • Proposition 1: Weak solution
  • Lemma A.1: Existence, uniqueness and interior regularity
  • proof
  • ...and 6 more