Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improving Surrogate Model Robustness to Perturbations for Dynamical Systems Through Machine Learning and Data Assimilation

Abhishek Ajayakumar, Soumyendu Raha

TL;DR

This work addresses the fragility of reduced-order surrogate models under input perturbations in high-dimensional dynamical systems on graphs. It introduces a framework that couples dynamic optimization to learn a sparse graph Laplacian with a POD-based surrogate, hierarchical clustering to diagnose model errors, an inverse problem to infer inter-cluster transitions, and Bayesian filtering to robustly estimate states. The approach yields a sparse, efficient Laplacian $L_1= B^T\text{diag}(w^*)B$ that enhances filtering performance and overall accuracy, demonstrated on linear diffusion and nonlinear reaction-diffusion dynamics, with additional benchmarking on neural ODE surrogates. The results indicate meaningful improvements in RMSE under perturbations and highlight practical potential for robust, scalable surrogate modeling in complex networked systems, while acknowledging computational overhead and divergence risks in certain configurations.

Abstract

Many real-world systems are modelled using complex ordinary differential equations (ODEs). However, the dimensionality of these systems can make them challenging to analyze. Dimensionality reduction techniques like Proper Orthogonal Decomposition (POD) can be used in such cases. However, these reduced order models are susceptible to perturbations in the input. We propose a novel framework that combines machine learning and data assimilation techniques to improving surrogate models to handle perturbations in input data effectively. Through rigorous experiments on dynamical systems modelled on graphs, we demonstrate that our framework substantially improves the accuracy of surrogate models under input perturbations. Furthermore, we evaluate the framework's efficacy on alternative surrogate models, including neural ODEs, and the empirical results consistently show enhanced performance.

Improving Surrogate Model Robustness to Perturbations for Dynamical Systems Through Machine Learning and Data Assimilation

TL;DR

This work addresses the fragility of reduced-order surrogate models under input perturbations in high-dimensional dynamical systems on graphs. It introduces a framework that couples dynamic optimization to learn a sparse graph Laplacian with a POD-based surrogate, hierarchical clustering to diagnose model errors, an inverse problem to infer inter-cluster transitions, and Bayesian filtering to robustly estimate states. The approach yields a sparse, efficient Laplacian that enhances filtering performance and overall accuracy, demonstrated on linear diffusion and nonlinear reaction-diffusion dynamics, with additional benchmarking on neural ODE surrogates. The results indicate meaningful improvements in RMSE under perturbations and highlight practical potential for robust, scalable surrogate modeling in complex networked systems, while acknowledging computational overhead and divergence risks in certain configurations.

Abstract

Many real-world systems are modelled using complex ordinary differential equations (ODEs). However, the dimensionality of these systems can make them challenging to analyze. Dimensionality reduction techniques like Proper Orthogonal Decomposition (POD) can be used in such cases. However, these reduced order models are susceptible to perturbations in the input. We propose a novel framework that combines machine learning and data assimilation techniques to improving surrogate models to handle perturbations in input data effectively. Through rigorous experiments on dynamical systems modelled on graphs, we demonstrate that our framework substantially improves the accuracy of surrogate models under input perturbations. Furthermore, we evaluate the framework's efficacy on alternative surrogate models, including neural ODEs, and the empirical results consistently show enhanced performance.
Paper Structure (17 sections, 3 theorems, 48 equations, 5 figures, 1 table, 1 algorithm)

This paper contains 17 sections, 3 theorems, 48 equations, 5 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Orthogonal collocation method
  4. Spatio temporal propagation in graphs
  5. Proper Orthogonal Decomposition for Dynamical systems
  6. Filtering
  7. Problem statement
  8. Proposed Framework
  9. Dynamic optimization problem formulation for dynamical systems represented on graphs
  10. Some known bounds on eigen values of Laplacian matrix
  11. Dynamic Optimization for Linear Dynamical Systems on Graphs Using a POD-Based Surrogate Model
  12. Dynamic Optimization for Reaction-diffusion Dynamical Systems on Graphs Using a POD-Based Surrogate Model
  13. Experimental Results
  14. Linear dynamical system represented on graphs
  15. Reaction-diffusion system represented on graphs
  16. ...and 2 more sections

Key Result

Theorem 1

Bernstein's Inequality: Suppose $X_1, \ldots, X_n$ are independent random variables with finite variances, and suppose that $\text{max}_{1 \leq i \leq n} \vert X_i \vert \leq B$ almost surely for some constant $B > 0$. Let $V = \sum_{i=1}^n \mathbb{E}X_i^2.$ Then, for every $t \geq 0,$ and

Figures (5)

  • Figure 1: Diagram illustrating the key steps of the proposed methodology, including dynamic optimization, hierarchical clustering, solving the inverse problem, and filtering. See Section \ref{['framework']} for a detailed explanation.
  • Figure 3: Comparison of the (a) ROM Solution, (b) Actual Solution, and (c) ROM solution with our framework for the graph-based dynamical system $\mathbb{D}$, as described in Section \ref{['spatio-temporal']}.
  • Figure 4: Comparing (a) ROM, (b) Actual, and (c) ROM solution with our framework for the case of the chemical Brusselator reaction-diffusion system (Eq. \ref{['Rd dynamics']}).
  • Figure 5: Comparison of (a) the actual solution, (b) the neural-ODE solution, (c) the neural-ODE solution with our framework, and (d) the ROM solution for a random initial condition in the case of a linear dynamical system on graphs ($\mathbb{D}$ in Section \ref{['spatio-temporal']}).
  • Figure 6: Comparison of absolute errors between the neural-ODE surrogate model with our framework, the standalone neural-ODE model, and the ROM solution. Black regions indicate grid points where the solutions given by the framework achieves a lower absolute error than the other methods.

Theorems & Definitions (5)

  • Theorem 1
  • Theorem 2
  • proof
  • Theorem 3
  • proof