The topology of synergy: linking topological and information-theoretic approaches to higher-order interactions in complex systems

TL;DR

The study tackles how to characterize higher-order interactions in complex systems by bridging topology and information theory. It compares topological data analysis (via cavities and persistence) with information-theoretic measures such as the O-information, total correlation $TC$, dual total correlation $DTC$, and S-information $\mathcal{S}$, using both synthetic manifolds and resting-state fMRI data. A key finding is that synergistic information is tied to three-dimensional cavities, and that intrinsic as opposed to contextual higher-order information behaves differently under rotations or projections; dimensionality reduction via PCA tends to preserve redundancies while suppressing synergies, revealing limitations of common low-dimensional analyses. The results suggest a path toward a unified theory spanning topology and information theory and highlight practical limits of prevalent methods for detecting higher-order structure in high-dimensional data, with implications for neuroscience and other complex systems.

Abstract

The study of irreducible higher-order interactions has become a core topic of study in complex systems. Two of the most well-developed frameworks, topological data analysis and multivariate information theory, aim to provide formal tools for identifying higher-order interactions in empirical data. Despite similar aims, however, these two approaches are built on markedly different mathematical foundations and have been developed largely in parallel. In this study, we present a head-to-head comparison of topological data analysis and information-theoretic approaches to describing higher-order interactions in multivariate data; with the aim of assessing the similarities and differences between how the frameworks define ``higher-order structures." We begin with toy examples with known topologies, before turning to naturalistic data: fMRI signals collected from the human brain. We find that intrinsic, higher-order synergistic information is associated with three-dimensional cavities in a point cloud: shapes such as spheres are synergy-dominated. In fMRI data, we find strong correlations between synergistic information and both the number and size of three-dimensional cavities. Furthermore, we find that dimensionality reduction techniques such as PCA preferentially represent higher-order redundancies, and largely fail to preserve both higher-order information and topological structure, suggesting that common manifold-based approaches to studying high-dimensional data are systematically failing to identify important features of the data. These results point towards the possibility of developing a rich theory of higher-order interactions that spans topological and information-theoretic approaches while simultaneously highlighting the profound limitations of more conventional methods.

Figures (10)

• Figure 1: Estimating underlying distributions via K-nearest neighbors measures. This cartoon demonstrates how discrete K-nearest neighbors analyses can be used to estimate the structure of a continuous, underlying distribution. Consider a bivariate normal distribution (left): if we sample a large number of points from it (center), we see that the density of the point cloud tracks the underlying local probability density around each point. If we then compute the distance to the fourth nearest neighbor, and color the points (right), we see how the the distribution of distances roughly recapitulates the underlying bivariate Gaussian.
• Figure 2: Kraskov mutual information estimator. A brief cartoon detailing the basic logic of the Kraskov mutual information estimator kraskov_estimating_2004. For each point, a diameter is defined by the distance from that point to it's fourth nearest neighbor. The estimator then counts the number of points within the diameter projected down to the constituent axes, and from that computes the local (pointwise) mutual information lizier_jidt_2014. The expected value over all the pointwise values gives the estimated mutual information.
• Figure 3: Rips filtration. Consider a two-dimensional point cloud arranged into a rough ring (the same cloud as used in Figure \ref{['fig:kraskov_explainer']}). Around each point, balls (blue circles) are expanded, and when the radii of two balls intersect, an edge is drawn between the points. When the diameter is low, the simplicial complex is disconnected, comprised on small, tree-like structures. As the balls expand, the simplicial complex becomes denser, and large-scale structures in the data are revealed (in this case, the central void). Eventually, the diameters will be so large that the whole complex will be densely connected and the void will close.
• Figure 4: O-information of point-clouds with known structure. Top row: Two point clouds that display only "contextual" higher-order information. The one-dimensional line, when embedded in a three-dimensional space, is highly redundant, but after rotation with a PCA, all higher-order information is obliterated, as all the information can be represented by one dimension. Similarly, the two-dimensional plane is synergy-dominated when embedded in a three-dimensional space, but also loses its higher-order structure after rotation with PCA. Middle row: Two shapes that have "intrinsic" synergy associated with three-dimensional cavities. The sphere is perfectly rotationally symmetric, and so no rotation changes the value of the O-information, while the toroid contains a mixture of contextual and higher-order information. Bottom row: Two shapes that contain intrinsic redundancy: a trefoil knot, and its generalization, the $p,q$-knot (p=5, q=3). These curves a locally line-like, but cannot be losslessly embeded in a lower-dimensional space.
• Figure 5: Sampling triads and computing measures of higher-order structure. A: Sets of three brain regions are sampled from the cerebral cortex and their associated BOLD time series are extracted. B: Those time series can be represented as a point cloud embedded in a three-dimensional space (as in varley_topological_2021, with each point encoding the joint state of each of the three cortical parcells at time t. C: That point-cloud can then be analyzed using persistance homology or multivariate information-theoretic measures.