Time-Lagged Recurrence: a data-driven method to estimate the predictability of dynamical systems

TL;DR

The paper tackles the challenge of state-dependent local predictability in high-dimensional, multiscale dynamical systems where forward operators are unknown or data are noisy, limiting Lyapunov-based global predictability. It introduces a purely data-driven local predictability index, time-lagged recurrence $\alpha_{\eta}$, built from recurrences around a reference state and forward recurrences after a horizon $\eta$, with exceedances defined via a quantile $q$ and excluded temporal correlations by a Theiler window $w$. This index yields a quantity in $[0,1]$ that estimates the probability that neighboring trajectories remain close to the reference trajectory after time $\eta$, enabling scale-dependent predictability analyses that reveal, for example, reduced predictability near phase-space transitions or in blocked regimes. The authors further extend the framework with a weighted version $\alpha_{\eta}^*$, connect it to information theory via entropy considerations, and propose a real-time proxy based on analogues, illustrating applications to Lorenz-63, Rössler, slow earthquakes, the double pendulum, ECG/EEG data, and Euro-Atlantic Z500 atmospheric circulation. Overall, $\alpha_{\eta}$ offers a robust, purely data-driven diagnostic for local predictability in high-dimensional systems and highlights scale- and state-dependent structure that is not captured by global indices.

Abstract

Nonlinear dynamical systems are ubiquitous in nature and they are hard to forecast. Not only they may be sensitive to small perturbations in their initial conditions, but they are often composed of processes acting at multiple scales. Classical approaches based on the Lyapunov spectrum rely on the knowledge of the dynamic forward operator, or of a data-derived approximation of it. This operator is typically unknown, or the data are too noisy to derive its faithful representation. Here we propose a new data-driven approach to analyze the local predictability of dynamical systems. This method, based on the concept of recurrence, is closely linked to the well-established framework of local dynamical indices. When applied to both idealized systems and real-world datasets arising from large-scale atmospheric fields, our new approach proves its effectiveness in estimating local predictability. Additionally, we discuss its relationship with other local dynamical indices, and how it reveals the scale-dependent nature of predictability. Furthermore, we explore its link to information theory, its extension that includes a weighting strategy, and its real-time application. We believe these aspects collectively demonstrate its potential as a powerful diagnostic tool for complex systems.

Figures (17)

• Figure 1: Schematic illustration of the computation of $\alpha_{\eta}(t)$, demonstrated in the phase space of the Lorenz-63 system. This figure presents all steps involved in computing the time-lagged recurrence for the reference state $\zeta$ at a forecasting horizon $\eta$, namely $\alpha_{\eta}(\zeta)$. The panels in the second row provide a zoomed-in view of the phase space region where $\alpha_{\eta}(\zeta)$ is defined. Recurrences ($R_{t_{\zeta}}$), forward recurrences ($R_{t_{\zeta}}^{\eta}$), and forward-reference-state recurrences ($R_{t_{\zeta}+{\eta}}$) are represented by solid blue dots, empty blue dots, and orange dots, respectively. The blue circle with radius $e^{-s(q, \zeta)}$ indicates the hypersphere used to define the neighborhood of the reference state, while the orange circle with radius $e^{-s(q, \zeta^\eta)}$ corresponds to the forward-reference-state.
• Figure 2: Distribution of $\alpha_{\eta}$ at different prediction horizons for the Lorenz-63 system. (a-e) Lorenz attractor colored by $\alpha_{\eta}$ values at five different forcasting horizons: $\eta = [0.05, 0.1, 0.2, 1, 2]\eta_{\ell}$, each corresponding to timesteps: $L = [11, 22, 44, 220, 440]$. (f) Probability distribution of $\alpha_{\eta}$ at the same five different forecasting horizons. The quantile $q$ applied in this analysis is 0.99, and the Theiler window size $w$ is set to 50 time steps.
• Figure 3: Temporal and phase-spatial variations of $\alpha_{\eta}$. The evolution of $\alpha_{\eta}$ averaged across seven clusters of states on the Lorenz-63 attractor. Inset: Lorenz-63 attractor colored to represent seven clusters obtained from K-means algorithm, each labeled with corresponding numbers.
• Figure 4: Comparative analysis of $\alpha_{\eta}$ with local dynamical indices for the Lorenz-63 system. (a-f) Scatter plots of local dimension versus local inverse persistence, where each dot represents one state. All states are colored based on $\alpha_{\eta}$ values for $\eta = [0.05, 0.1, 0.2, 1, 2, 4]\eta_{\ell}$. Note that the color scale is different in the last three panels in order to improve readability.
• Figure 5: Predictability analysis of North Atlantic wintertime weather regimes. (a-c) Probability distribution of $\alpha_{\eta}$ for four weather regime and no regime: NAO+ (blue dashed line), Atlantic Ridge (red dotted line), NAO- (green dot-dashed line), Scandinavian Blocking (magenta loosely dashed line), and no regime (black line), for $\eta=1-3$ days. The markers on the x-axis show the mean value of $\alpha_{\eta}$ for different categories. The statistical significance is evaluated between the distribution of each weather regime and that of no regime using the Kolmogorov-Smirnov test. The quantile $q$ applied in this analysis is 0.99, and the Theiler window size $w$ is set to 7 days.