Characterising high-order interdependence via entropic conjugation

TL;DR

This work tackles the landscape of high-order interdependence metrics built from Shannon entropy by introducing entropic conjugation, a principled duality that maps high- to low-order interactions and redundancy–synergy balance. It shows that many existing metrics are related through this conjugation and proves that symmetric and skew-symmetric components form natural, orthogonal directions in the space of interdependencies, with explicit relationships for metrics such as TC, DTC, Σ, Ω, II, and TSE. Up to n=5 variables, the known metrics span the symmetric and skew-symmetric subspaces, while S-information and O-information emerge as the most computationally efficient representatives of these classes. Empirically, spin-system data reveal that two principal components—one symmetric and one skew-symmetric—capture almost all variability, supporting a two-axis description of high-order interdependence strength and balance across physical systems.

Abstract

High-order phenomena play crucial roles in many systems of interest, but their analysis is often highly nontrivial. There is a rich literature providing a number of alternative information-theoretic quantities capturing high-order phenomena, but their interpretation and relationship with each other is not well understood. The lack of principles unifying these quantities obscures the choice of tools for enabling specific type of analyses. Here we show how an entropic conjugation provides a theoretically grounded principle to investigate the space of possible high-order quantities, clarifying the nature of the existent metrics while revealing gaps in the literature. This leads to identify novel notions of symmetry and skew-symmetry as key properties for guaranteeing a balanced account of high-order interdependencies and enabling broadly applicable analyses across physical systems.

Figures (1)

• Figure 1: Information-theoretic analysis of the interdependencies observed in systems of $n=8$ spins subject to positive (ferromagnetic), negative (frustrated), and weak interactions between them. a) When calculating $u_k$, each type of interactions exhibit distinct profiles of interdependence. b) The variability among $u_k$ is captured by two principal components: one of symmetric character which accounts for the overall strength of the interdependence (PC1), and one of skew-symmetric character that accounts for the balance between high- and low-order interdependence (PC2). c) The values of $u_k$ projected onto these PCs provide a simple characterisation of these three types of systems in terms of their overall interdependence strength (PC1) and the balance between high- and low-order effects (PC2).

Theorems & Definitions (21)

• Definition 1
• Proposition 1
• Theorem 1
• Definition 2
• Lemma 1
• Proposition 2
• Theorem 2
• Corollary 1
• Proposition 3
• proof : Proof of Prop. \ref{['prop:u_conj']}