Synergy as the failure of distributivity

TL;DR

The paper tackles the lack of a rigorous definition for emergence by linking information theory to a distributivity-free set-theoretic framework. It shows that synergy arises as a consequence of distributivity breaking when analyzing multiple random variables, and introduces information atoms (including a novel synergy and ghost atom) that yield a nonnegative multivariate decomposition. Using the XOR gate as a canonical example, it derives a Venn-like diagram and extends to general tri-variate and N-variable cases, including N-parity, while addressing prior PID self-contradictions. The work lays a foundation for a self-consistent multivariate information decomposition and posits non-distributive variants of set theory as a natural language for describing emergent physical systems, with potential applications to quantifying emergence in complex systems.

Abstract

The concept of emergence, or synergy in its simplest form, is widely used but lacks a rigorous definition. Our work connects information and set theory to uncover the mathematical nature of synergy as the failure of distributivity. It resolves the persistent self-contradiction of information decomposition theory and reinstates it as a primary route toward a rigorous definition of emergence. Our results suggest that non-distributive variants of set theory may be used to describe emergent physical systems.

Figures (4)

• Figure 1: A diagram representing different types of mutual information in the system of two random variables $X, Y$ about $Z$ by the part-whole relations. Redundant information is shared between $X$ and $Y$, unique information is a part of just one of them, while synergistic information is something that is only contained in the joint distribution, but not individual sources on their own.
• Figure 2: A Venn-type diagram for the XOR gate. Each variable is represented by a primary color circle (red, yellow, blue) while the outer circle outlines the whole system. Of the total $2$ bits of the XOR gate, one is covered two times and is represented by the inner disk. Since it is covered twice this area is colored by pairwise color-blends (orange, purple, and green). Since it is covered by three variables it includes patches of all three possible blends. A critical difference between this diagram and a set-theoretic one is that even though the three variables have no pairwise intersections, the inner disk representing the 'mutual' content of all three variables is non-empty. The remaining $1$ bit is covered once and resides only inside the joint distribution. Since this area is covered once, it is colored by primary colors. Patches of all three colors are used since this area does not belong to any single variable.
• Figure 3: A single realization of inclusion-exclusion principle for three variables. The new region, corresponding to the distributivity-breaking difference is represented via a checkered pattern. Covering numbers are written for each sector and highlighted by the colors. This is not a full Venn-type diagram that defines the information atoms, hence its structure is clearly not invariant with respect to variable permutations.
• Figure 4: A graphical illustration for the general solution of the trivariate problem. Compared to the Venn diagram for $3$ sets, two new regions here are the $2$-covered part of triple intersection $\Pi_s$ (synergistic atom) and a ghost atom $\Pi_g$, which is not a part of any single initial variable. Similarly to Fig. \ref{['fig:3varInfoSpace']}, colors indicate the coverings: 3 primary colors (red, yellow, blue or their checkered combination) correspond to 1-covered atoms, the overlay of any 2 colors (orange, purple, green or their checkered combination) is 2-covered and the overlay of all 3 colors (brown) is 3-covered.

Theorems & Definitions (17)

• Lemma 1
• proof
• Corollary
• Lemma 2
• proof
• Theorem 1
• proof
• Lemma 3
• Lemma 4
• proof