TENDE: Transfer Entropy Neural Diffusion Estimation

TL;DR

TE measures directed information flow in time series via $TE_{X \to Y}(k,l) = I(Y_t; \mathbf{X}_{t-k} | \mathbf{Y}_{t-l})$, but traditional estimators struggle in high dimensions. TENDE introduces a score-based diffusion framework that learns conditional score functions to estimate TE through conditional mutual information, offering flexibility with minimal distributional assumptions. Empirical results on synthetic benchmarks and real data show TENDE achieving high accuracy and robustness, even as dimensionality grows, and outperforming several state-of-the-art estimators. This approach enables reliable analysis of directional information flow in complex dynamical systems with practical impact in neuroscience, finance, and beyond.

Abstract

Transfer entropy measures directed information flow in time series, and it has become a fundamental quantity in applications spanning neuroscience, finance, and complex systems analysis. However, existing estimation methods suffer from the curse of dimensionality, require restrictive distributional assumptions, or need exponentially large datasets for reliable convergence. We address these limitations in the literature by proposing TENDE (Transfer Entropy Neural Diffusion Estimation), a novel approach that leverages score-based diffusion models to estimate transfer entropy through conditional mutual information. By learning score functions of the relevant conditional distributions, TENDE provides flexible, scalable estimation while making minimal assumptions about the underlying data-generating process. We demonstrate superior accuracy and robustness compared to existing neural estimators and other state-of-the-art approaches across synthetic benchmarks and real data.

Figures (7)

• Figure 1: Transfer entropy estimation across sample sizes for linear Gaussian and joint systems.
• Figure 2: Transfer entropy estimation for varying coupling strength ($\lambda$).
• Figure 3: Transfer entropy estimation with added redundant (noise) dimensions.
• Figure 4: Transfer entropy estimation for linearly stacked systems where multiple independent process copies create additive transfer entropy.
• Figure 5: The top row displays the sampled heartbeat and respiration force time series from the Santa Fe dataset shown over 10 minutes. The bottom row shows the transfer entropy $TE(k,\ell=2)$ between breathing and heart signals as a function of lag $k$, showing directional information flow from one signal to another. The reported error bars correspond to the standard deviations over 5 seeds.