Considering dynamical synergy and integrated information; the unusual case of minimum mutual information

Abstract

This brief note considers the problem of estimating temporal synergy and integrated information in dyadic dynamical processes. One of the standard estimators of dynamic synergy is based on the minimal mutual information between sets of elements, however, despite it's increasingly widespread use, the mathematical features of this redundancy function have largely gone unexplored. Here, we show that it has two previously unrecognized limitations: it cannot disambiguate between truly integrated systems and disintegrated systems with first-order autocorrelation. Second, paradoxically, there are some systems that become more synergistic when dis-integrated (as long as first-order autocorrelations are preserved). In these systems, integrated information can decrease while synergy simultaneously increases. We derive conditions under which this occurs and discuss the implications of these findings for past and future work in applied fields such as neuroscience.

Figures (2)

• Figure 1: Left: The transition probability matrix for system X. This system is disintegrated: $X^1\bot X^2$, however, each individual process contains 1 bit of autocorrelation, and so the predictive power of the whole is trivially reducible to the sum of its parts. Right: The transition probability matrix for system Y. This system is "integrated", in that there is 1 bit of temporal mutual information in the whole, but the individual elements contain no predictive power about their own futures.
• Figure 2: Top left: We find a strong, positive relationship between the change in total temporal mutual information upon disintegration and the change in synergy. Systems that show an increase in temporal mutual information also tend to show an increase in synergy. Top right: As expected given the derivation of Equation \ref{['eq:delta_2']}, the correlation between $X^1$ and $X^2$ is a significant driver how the apparent synergy in $I(\textbf{X}_t;\textbf{X}_{t+1})$ changes after disintegration. The greater the functional connectivity, the more synergy is produced when doing the circular shifts. Bottom left: The relationship between functional connectivity and global increase in temporal mutual information upon disintegration is strong, but not as strong as the change in synergy. Bottom right: This plot illustrates the central paradox discussed here: after disintegration, the integrated information $\Phi^{WMS}(\textbf{X})$ goes to zero, however in many pairs of brain regions, the MMI-synergy increases in a predictable, approximately linear way. Clearly synergy and integrated information are behaving in profoundly different ways.