Information Flow Rate for Cross-Correlated Stochastic Processes

Abstract

Causal inference seeks to identify cause-and-effect interactions in coupled systems. A recently proposed method by Liang detects causal relations by quantifying the direction and magnitude of information flow between time series. The theoretical formulation of information flow for stochastic dynamical systems provides a general expression and a data-driven statistic for the rate of entropy transfer between different system units. To advance understanding of information flow rate in terms of intuitive concepts and physically meaningful parameters, we investigate statistical properties of the data-driven information flow rate between coupled stochastic processes. We derive relations between the expectation of the information flow rate statistic and properties of the auto- and cross-correlation functions. Thus, we elucidate the dependence of the information flow rate on the analytical properties and characteristic times of the correlation functions. Our analysis provides insight into the influence of the sampling step, the strength of cross-correlations, and the temporal delay of correlations on information flow rate. We support the theoretical results with numerical simulations of correlated Gaussian processes.

Figures (5)

• Figure 1: Auto- and cross-covariance functions (top) and simulated samples (bottom) of two Gaussian processes representing a driver time series $X_{1}(t)$ (continuous line, blue online) and a receiver time series $X_{2}(t)$ (broken line, red online). The plots are based on the bivariate Gaussian process with the delayed square exponential covariance model \ref{['eq:cova-gaussian-simul']}. In the top frame, the auto-covariance is marked by the continuous line (cyan online), the cross-covariance $C_{1,2}(\tau)$ by the broken line (red online), and the cross-covariance $C_{2,1}(\tau)$ by the dash-dot line (blue online).
• Figure 2: Histograms of continuous sampling IFR $\mathcal{T}_{1 \to 2}$ (top) and $\mathcal{T}_{2 \to 1}$ (bottom) generated from an ensemble of 100 realizations of a bivariate cross-correlated Gaussian stochastic process with square exponential covariance model \ref{['eq:cova-gaussian-simul']}. The continuous line in the middle (red online) of both plots marks the theoretical estimate of the equilibrium IFR \ref{['eq:tij-time-delay-gauss']}. The model parameters for the stochastic processes (cf. Section \ref{['ssec:gauss-time-delay-simul']}), are: $T=10$ (length of sampling window) ${\delta t}=0.002$ (sampling step), $N=5\times 10^3$ (number of sampling points), $\tau_{1}=\tau_{2}=\tau_{0}=0.05$ (correlation time), $\tau_{\ast}=0.008$ (time delay), $\sigma_{1}^{2}=\sigma_{2}^{2}=1.1$ and $\sigma_{0}^{2}=1$.
• Figure 3: Parametric plots of ${\mathcal{T}}_{1\to2}$ for the regression model $X_{2}(t)=a X_{1}(t-\tau_{\ast})+ Y(t)$ (see Section \ref{['ssec:msd-regression-delayed']}) versus $\tau_{\ast}/\tau_{0}$ for different $b/a$ ratios. $X_{1}(t), X_{2}(t)$ and $Y(t)$ are Gaussian processes with square exponential auto- and cross-covariances. It is assumed that $\sigma^{2}_{0}=1$, $\sigma^{2}_{1}=1.1$, $a=\sigma^{2}_{0}/\sigma^{2}_{1}$ and $\sigma_{2}^{2}= a^{2}\,\sigma_{1}^{2} (1 +b^{2}/a^{2})$. The time constants are $T=100$, $\tau_{1}=\tau_{2}=\tau_{0}=1$, while the time delay $\tau_{\ast}$ is determined from the ratio $\tau_{\ast}/\tau_{0}$. The continuous curves are based on the theoretical equilibrium IFR expression \ref{['eq:Tij_differ_delayed']}. For every realization (pair of length $N=1000$ time series) from an ensemble of $N_{\mathrm{sim}}=100$ simulations, the IFR is estimated based on the data-driven estimator \ref{['eq:tij']}. Lower values of $b/a$ imply higher cross correlation between $X_1$ and $X_2$. The star markers denote ensemble averages of IFR estimates, while the associated error bars have a width of two standard deviations (as estimated from the ensemble).
• Figure 4: Histograms of continuous sampling IFR $\mathcal{T}_{1 \to 2}$ (top) and $\mathcal{T}_{2 \to 1}$ (bottom) generated from an ensemble of 100 realizations comprising two Gaussian stochastic processes governed by the time-delayed, exponential (Ornstein-Uhlenbeck) model which is defined in Section \ref{['ssec:msc-O-U']}. The vertical lines in the middle of the histograms (red online) near 60 and 0 respectively, mark the theoretical, equilibrium IFR estimate \ref{['eq:tij-time-delay-expon']}. The model parameters for the stochastic processes are: $T=10$ (length of sampling window) ${\delta t}=0.002$ (sampling step), $N=5\times 10^3$ (number of sampling points), $\tau_{1}=\tau_{2}=\tau_{0}=0.05$ (correlation time), $\tau_{\ast}=0.008$ (time delay), $\sigma_{1}^{2}=\sigma_{2}^{2}=1.1$ and $\sigma_{0}^{2}=1$.
• Figure 5: Parametric plots (continuous curves) of $T_{1\to2}$ (top) and $T_{2\to1}$ (bottom) versus $\tau_{\ast}/\tau_{0}$ for different $b/a$ ratios. Two Gaussian processes, $X_{1}$ and $X_{2}$, with time-delayed, Ornstein-Uhlenbeck model---defined in \ref{['eq:cross-o-u-delayed']}, are simulated. It is assumed that $\sigma^{2}_{0}=1$, $\sigma^{2}_{1}=1.1$, $a=\sigma^{2}_{0}/\sigma^{2}_{1}$ and $\sigma_{2}^{2}= a^{2}\,\sigma_{1}^{2} (1 +b^{2}/a^{2})$. Lower $b/a$ values imply higher cross correlation between $X_{1}$ and $X_{2}$. The time constants are $T=100$, $\tau_{1}=\tau_{2}=\tau_{0}=1$, while $\tau_{\ast}$ is determined from the ratio $\tau_{\ast}/\tau_{0}$. The continuous curves are based on the theoretical equilibrium estimates, i.e., \ref{['eq:tij-time-delay-expon']} for ${\delta t} < \tau_{\ast}$ (top and bottom panels), and \ref{['eq:ifr-expon-small-delay']} for $T_{1\to2}$ if ${\delta t} \ge \tau_{\ast}$ (top). For every realization (pair of length $N=1000$ time series) from an ensemble of $N_{\mathrm{sim}}=100$ simulations, the IFR is obtained using the data-driven estimator \ref{['eq:tij']}. Ensemble-based averages are marked by stars, while the associated error bars have a width of two standard deviations (based on the ensemble estimates).

Theorems & Definitions (16)

• Definition 1: Vector stochastic process
• Definition 2: Auto- and cross-covariance functions
• Definition 3: Time series
• Definition 4: Stationarity
• Definition 5: Sample covariance functions
• Definition 6: Nonnegative definiteness
• Definition 7: Fourier transforms
• Remark 1
• Definition 8: Separable cross-correlation model
• Remark 2: General properties of $\hat{T}_{i \to j}$