Evolving higher-order synergies reveals a trade-off between stability and information integration capacity in complex systems

TL;DR

There may be a fundamental trade-off between the robustness of a system's dynamics and its capacity to integrate information and that certain kinds of complexity naturally balance this trade-off.

Abstract

There has recently been an explosion of interest in how "higher-order" structures emerge in complex systems. This "emergent" organization has been found in a variety of natural and artificial systems, although at present the field lacks a unified understanding of what the consequences of higher-order synergies and redundancies are for systems. Typical research treat the presence (or absence) of synergistic information as a dependent variable and report changes in the level of synergy in response to some change in the system. Here, we attempt to flip the script: rather than treating higher-order information as a dependent variable, we use evolutionary optimization to evolve boolean networks with significant higher-order redundancies, synergies, or statistical complexity. We then analyse these evolved populations of networks using established tools for characterizing discrete dynamics: the number of attractors, average transient length, and Derrida coefficient. We also assess the capacity of the systems to integrate information. We find that high-synergy systems are unstable and chaotic, but with a high capacity to integrate information. In contrast, evolved redundant systems are extremely stable, but have negligible capacity to integrate information. Finally, the complex systems that balance integration and segregation (known as Tononi-Sporns-Edelman complexity) show features of both chaosticity and stability, with a greater capacity to integrate information than the redundant systems while being more stable than the random and synergistic systems. We conclude that there may be a fundamental trade-off between the robustness of a systems dynamics and its capacity to integrate information (which inherently requires flexibility and sensitivity), and that certain kinds of complexity naturally balance this trade-off.

Figures (3)

• Figure 1: Intervention distribution. The state-transition structure of a boolean network defines a transition probability matrix (TPM), which gives the probability of a given output (the columns), conditional on a given input (the rows). Here we see the TPM for a three-element system X, and how the intervention distribution is computed. At time $t$, all global states are equally likely, corresponding to a maximum-entropy distribution. After one timestep, to time $t+1$, the distribution of possible states is no longer uniform: the is the distribution of states after performing an intervention on Xhoel_quantifying_2013 - it is this distribution that is fed into the calculations for O-information and Tononi-Sporns-Edelman complexity.
• Figure 2: Evolutionary optimization of redundancy, synergy, and complexity. Top row: Presented above are the evolutionary trajectories for all populations evolving for redundancy (left), synergy (middle), and TSE complexity (right). From random initial conditions, it is clear that the evolutionary optimization is able to discover many configurations that significantly deviate from the random initial conditions. This figure establishes the colour schema that will be used for the paper. Red for redundancy, blue for synergy, and gold for TSE complexity. Bottom row: For each of the three classes of system (high redundancy, high synergy, and high complexity), we selected fittest boolean networks and ran them to get a visual sense of their different properties. Note how the highly-redundant system almost immediately falls into a stable attractor, while the high-synergy system has a long transient time and overall visually noisier patterns. While these individual trajectories are one of many, they are representative of broader trends. For the redundant system, the average tansient time required to hit an attractor is merely 2.75 steps. In contrast, for the synergistic system, the average transient time is 26.91 steps, and for the TSE-maximizing system, the average transient time is 7.7 steps.
• Figure 3: Dynamical differences between random, synergistic, redundant, and complex systems. Top right: The number of unique attractors for each network, for each class. Top left: The joint entropy of the global state intervention distribution after one timestep. Middle left: The Derrida coefficient for each network, for each class. Middle right: The average length of the transients. Bottom: The integrated information capacity $\Phi^{R}$ for each network for each class.