A scalable, synergy-first backbone decomposition of higher-order structures in complex systems

TL;DR

An alternative decomposition that is synergy-first, scales much more gracefully than the PID, and has a straightforward interpretation is presented, and it is shown that this approach can be used to decompose higher-order interactions beyond information theory by showing how synergistic combinations of edges in a graph support global integration via communicability.

Abstract

Since its introduction in 2011, the partial information decomposition (PID) has triggered an explosion of interest in the field of multivariate information theory and the study of emergent, higher-order ("synergistic") interactions in complex systems. Despite its power, however, the PID has a number of limitations that restrict its general applicability: it scales poorly with system size and the standard approach to decomposition hinges on a definition of "redundancy", leaving synergy only vaguely defined as "that information not redundant." Other heuristic measures, such as the O-information, have been introduced, although these measures typically only provided a summary statistic of redundancy/synergy dominance, rather than direct insight into the synergy itself. To address this issue, we present an alternative decomposition that is synergy-first, scales much more gracefully than the PID, and has a straightforward interpretation. Our approach defines synergy as that information in a set that would be lost following the minimally invasive perturbation on any single element. By generalizing this idea to sets of elements, we construct a totally ordered "backbone" of partial synergy atoms that sweeps systems scales. Our approach starts with entropy, but can be generalized to the Kullback-Leibler divergence, and by extension, to the total correlation and the single-target mutual information. Finally, we show that this approach can be used to decompose higher-order interactions beyond just information theory: we demonstrate this by showing how synergistic combinations of pairwise edges in a complex network supports signal communicability and global integration. We conclude by discussing how this perspective on synergistic structure (information-based or otherwise) can deepen our understanding of part-whole relationships in complex systems.

Figures (2)

• Figure 1: The bipartition approach to local $\boldsymbol{\alpha}$-synergistic entropy.Top row: For a seven-element system X in a particular state (x=(1,0,0,1,0,1,1)), the $\alpha$-synergistic entropy decomposition requires partitioning x into all possible subsets of size $\alpha$, and finding the minimum value of $h(\textbf{x}^{\textbf{a}}|\textbf{x}^{-\textbf{a}})$. Bottom row: by sweeping all scales $1\ldots k$, it is possible to construct a hierarchical picture of how fragile synergy is distributed over scales.
• Figure 2: Communicability example: For a randomly generated Erdos-Renyi graph with ten nodes, and edge weights pulled from an exponential distribution with $\lambda=1$, we compute the $\alpha$-synergy spectrum for the communicability across all possible partitions of the edge set, and compute the $\alpha$-synergy and $\partial$-synergy for each one. On the leftmost panel is the weighted adjacency matrix, with the associated network in he middle panel. In the rightmost panel, we plot the expected $\alpha$-synergistic communicability against the number of failing edges, with the points coloured by the partial synergistic communicability.