From description to design: Automated engineering of complex systems with desirable emergent properties

Abstract

The study of complex systems has produced a huge library of different descriptive statistics that scientists can use to describe the various emergent patterns that characterize complex systems. The problem of engineering systems to display those patterns from first principles is a much harder one, however, as a hallmark of complexity is that macro-scale emergent properties are often difficult to predict from micro-scale features. Here, we propose a general optimization-based pipeline to automate the difficult problem of engineering emergent features by re-purposing descriptive statistics as loss functions, and letting a gradient descent optimizer do the hard work of designing the relevant micro-scale features and interactions. Using Kuramoto systems of coupled oscillators as a test bed, we show that our approach can reliably produce systems with non-trivial global properties, including higher-order synergistic information, multi-attractor metastability, and meso-scale structures such as modules and integrated information. We further show that this pipeline can also account for and accommodate constraints on the system properties, such as the costs of connections, or topological restrictions. This work is a step forward on the path moving complex systems science from a field predicated largely on description and post-hoc storytelling towards one capable of engineering real-world systems with desirable emergent meso-scale and macro-scale properties.

Figures (7)

• Figure 1: The standard approach in systems science: the system and the observer are assumed to be independent. An observer collects data from a system, which is then analyzed using techniques from network science, statistics, machine learning, or any other field of science. These analyses result in a description (or alternately, a model) of the system, from which insights into the nature of the original system can be derived. Crucially, the flow from system to description is strictly one-way: the acts of observation and description have no impact on the thing been observed or described.
• Figure 2: The design approach
• Figure 3: Batched optimization of Kuramoto models. A batch of Kuramoto models are spawned, parametrized by a common connectivity matrix $\mathbf{A}$ and intrinsic frequencies $\mathbf{\omega}$, and initialized with random phases on each oscillator. The models are allowed to run, and phase time series are collected, and analyzed for global features (e.g. metastability, dual total correlation, etc). Those features are averaged, and the loss backpropagated to the initial parameters the defined the systems.
• Figure 4: Optimizing global features. A: The optimization curve for 100 Kuramoto models building high global dual total correlation. B: The optimization curve for 100 Kuramoto models building high negative O-information (indicating global synergy-dominance). C: The optimization curve for 100 models building high metastability (variance of the Kuramoto order parameter). D: The percentage change from initial dual total correlation before and after perturbation. E: The percentage change from initial negative O-information before and after perturbation. Note that, while perturbation does decrease the O-information, the post-perturbation system remains significantly more synergy-dominated than the initial conditions. C: The percentage change from initial metastability before and after perturbation. The metastability measure as the most fragile, although the post-perturbation values were still, on average, double the initial values.
• Figure 5: Optimized coupling matrices. A: The initial coupling strengths were randomly sampled from an exponential distribution with scale of 0.01, to produce noisy, but weak, coupling. B: The coupling matrix of the best-performing system optimized for dual total correlation. Note the increase in sparseness, with a smaller number of much larger edge driving the oscillator-to-oscillator coupling. C: The coupling matrix for the best-performing negative O-information. D: The best performing coupling matrix for the systems optimized for metastability. In all systems, there is clearly an emergence of "structure", but not one that is obviously informative or could be easily designed from first-principles by a human engineer.