CEBoosting: Online Sparse Identification of Dynamical Systems with Regime Switching by Causation Entropy Boosting

TL;DR

CEBoosting tackles online identification of regime switching in nonlinear dynamical systems by leveraging causation entropy indicators to selectively refine a sparse residual model, then fitting coefficients with closed-form least squares. It decouples structure discovery from parameter estimation and aggregates information across data batches to robustly detect regime changes with limited data. The approach is demonstrated on Lorenz 63/96, a topographic model, and a SPEKF system, showing effective detection, sparsity-based model correction, and compatibility with partial observations and extreme events. This yields a practical, computationally efficient framework for real-time identification and adaptation of complex dynamical systems.

Abstract

Regime switching is ubiquitous in many complex dynamical systems with multiscale features, chaotic behavior, and extreme events. In this paper, a causation entropy boosting (CEBoosting) strategy is developed to facilitate the detection of regime switching and the discovery of the dynamics associated with the new regime via online model identification. The causation entropy, which can be efficiently calculated, provides a logic value of each candidate function in a pre-determined library. The reversal of one or a few such causation entropy indicators associated with the model calibrated for the current regime implies the detection of regime switching. Despite the short length of each batch formed by the sequential data, the accumulated value of causation entropy corresponding to a sequence of data batches leads to a robust indicator. With the detected rectification of the model structure, the subsequent parameter estimation becomes a quadratic optimization problem, which is solved using closed analytic formulae. Using the Lorenz 96 model, it is shown that the causation entropy indicator can be efficiently calculated, and the method applies to moderately large dimensional systems. The CEBoosting algorithm is also adaptive to the situation with partial observations. It is shown via a stochastic parameterized model that the CEBoosting strategy can be combined with data assimilation to identify regime switching triggered by the unobserved latent processes. In addition, the CEBoosting method is applied to a nonlinear paradigm model for topographic mean flow interaction, demonstrating the online detection of regime switching in the presence of strong intermittency and extreme events.

Figures (9)

• Figure 2.1: Schematic of CEBoosting algorithm (based on the Lorenz 63 model). Panel (a): data is generated from the Lorenz 63 system with regime switching. The parameter in Regime 1 is $\sigma=10, \beta=8/3, \rho=28$, while $\rho$ is changed to 38 in Regime 2. The index of incoming batch data $k$ starts with $1$. Panel (b): $\boldsymbol{\Xi}^{(0)}$ is the current model parameter matrix defined in \ref{['eq:discretized_system']} with $\boldsymbol{\Phi}$ is the basis functions and $\dot{x}_i$ is the dynamic of state $x_i$. With $\boldsymbol{\Xi}^{(0)}$, the residual dynamics $r_i$ and the causation entropy between $r_i$ and $\boldsymbol{\Phi}$ can be calculated for each batch. $\text{CEM}(k)$ is the aggregated causation entropy from batch $1$ to $k$. $\mathbf{C}^+(k)$ is the binary matrix of $\text{CEM}(k)$ defined in \ref{['eq:C+']} indicating the sparse structure of the residual model. $D$ is defined in \ref{['eq:C_criterion']} indicating number that $\mathbf{C}^+(k)$ becomes stable, i.e., the pattern is consistent with the previous $D$ aggregated causation entropy matrix. Panel (c): with $\mathbf{C}^+(k)$ and data batches from the batch with new regime detected to the one with a stable $\mathbf{C}^+$, the residual model is calibrated by least square estimation.
• Figure 3.1: Lorenz 63 system with regime switching. (a): trajectories of the original system and the new one in phase space. (c): time series of system state variables. (b) and (d): autocorrelation function (ACF) of the original system and the new regime. (e): ensemble mean of variable $z$ before and after regime switching at $t=100$.
• Figure 3.2: The Lorenz 96 system with regime switching. Top: forty-dimensional system state evolving at the time interval $[0, 200]$ with a regime switching at $t=100$. Regime 1 is chaotic with $F=8$, and regime 2 is turbulent with $F=16$ and the linear terms $-1.5x_j$. Bottom: zoom-in view of the regime switching at the time interval $[90, 110]$.
• Figure 3.3: Statistical properties of the state variable $x_{10}$ of the Lorenz 96 system with regime switching. Panel (a): time series of $x_{10}$ in regime 1 and regime 2. Panels (b) and (c): the PDFs of $x_{10}$ in both regimes. Panels (d) and (e): the ACFs of $x_{10}$ in both regimes. Panels (f) and (g): the ensemble mean and the ensemble variance of $x_{10}$ before and after the regime switching at $t=100$.
• Figure 3.4: Topographic model with regime switching. Middle column: time series of each state in the time interval [0, 5000] with regime switching at $t=2500$. 1st and 5th columns: probability density function (PDF) of each state in both regimes. 2nd and 4th columns: autocorrelation function (ACF) of each state in both regimes.