Some Thoughts on Symbolic Transfer Entropy

TL;DR

This paper tackles the computational burden of Symbolic Transfer Entropy (STE) caused by the combinatorial growth of permutations with embedding dimension $m$ and sequence length $N$, situating STE as a robust measure of information flow via permutation-based encoding. It introduces two optimization strategies: Binning STE, which reduces permutations to $b^m$ and complexity $O(b^m)$, and Principal STE, which uses $t$ extreme-value groups with permutations $\dfrac{m!}{(m-2t)!}$, yielding $O(m^{2t})$ complexity while preserving main dynamics. Through experiments on New York precipitation and temperature data, the authors demonstrate that moderate bin counts ($b=5$ or $6$) and a small number of extreme groups ($t=2$ or $3$) can maintain estimation accuracy (as shown by declining MSE values) with substantial computational savings. However, the study notes limitations due to dataset size and scope, and suggests avenues like Dynamic Time Warping, wavelet transforms, and pattern-based methods for future work to further enhance estimation efficiency and reliability.

Abstract

Transfer entropy is used to establish a measure of causal relationships between two variables. Symbolic transfer entropy, as an estimation method for transfer entropy, is widely applied due to its robustness against non-stationarity. This paper investigates the embedding dimension parameter in symbolic transfer entropy and proposes optimization methods for high complexity in extreme cases with complex data. Additionally, it offers some perspectives on estimation methods for transfer entropy.

Figures (8)

• Figure 1: permutations of (0,1,2,3)
• Figure 2: Binning
• Figure 3: New York precipitation and temperature, with the lower graph showing a magnified view of the upper graph
• Figure 4: Comparison of Binning STE
• Figure 5: Comparison of permutation numbers for extreme cases in Binning STE