Information Filtering Networks: Theoretical Foundations, Generative Methodologies, and Real-World Applications

TL;DR

This paper surveys Information Filtering Networks (IFNs), a class of globally sparse yet locally dense higher-order networks for capturing multivariate dependencies. It details theoretical foundations, constructive algorithms (MST, PMFG, TMFG, MFCF), and how IFNs yield simplicial structures that support scalable, interpretable modeling across diverse domains. The review covers IFN-based covariance filtering (LoGo), feature selection, graphical modeling, and integration with Graph Neural Networks and novel architectures like Homological Neural Networks (HNN/HCNN), highlighting practical gains in interpretability and efficiency. Across finance, biology, climate science, psychology, neuroscience, and AI, IFNs enable robust dependency discovery, state-aware modeling, and scalable computation, while outlining open challenges in dynamics, scalability, and gain-function design for future research.

Abstract

Information Filtering Networks (IFNs) provide a powerful framework for modeling complex systems through globally sparse yet locally dense and interpretable structures that capture multivariate dependencies. This review offers a comprehensive account of IFNs, covering their theoretical foundations, construction methodologies, and diverse applications. Tracing their origins from early network-based models to advanced formulations such as the Triangulated Maximally Filtered Graph (TMFG) and the Maximally Filtered Clique Forest (MFCF), the paper highlights how IFNs address key challenges in high-dimensional data-driven modeling. IFNs and their construction methodologies are intrinsically higher-order networks that generate simplicial complexes-structures that are only now becoming popular in the broader literature. Applications span fields including finance, biology, psychology, and artificial intelligence, where IFNs improve interpretability, computational efficiency, and predictive performance. Special attention is given to their role in graphical modeling, where IFNs enable the estimation of sparse inverse covariance matrices with greater accuracy and scalability than traditional approaches like Graphical LASSO. Finally, the review discusses recent developments that integrate IFNs with machine learning and deep learning, underscoring their potential not only to bridge classical network theory with contemporary data-driven paradigms, but also to shape the architectures of deep learning models themselves.

Figures (9)

• Figure 1: Illustration of the network representation of the composible function $f(x_1, x_2, x_3, x_4) = h(h_a(x_1, x_2), h_b(x_2, x_3), h_c(x_4))$.
• Figure 2: A simple example of a chordal graph made of 6 vertices, three cliques, and two separators. It is a clique tree, a hypergraph with the cliques as hipervertices, and the separators as hyperedges.
• Figure 3: Examples of MFCF networks constructed to maximize the sum of correlations squared. The three networks on the top are trees, while the three on the bottom are planar graphs. MFCF(2,2,1), has max and min clique sizes equal to 2, and separators can be used only once. The result can only be a line. MFCF(2,2,2), also has max and min clique sizes equal to 2, but vertices can have coordination up to three. MFCF(2,2,$\infty$), is the maximum spanning tree, with max and min clique sizes equal to 2, and vertices with arbitrary coordination ($\infty$ indicates arbitrary coordination, in this case up to $p-1$). MFCF(3,3,1), introduces triangular cliques and separators are edges have max coordination 2. MFCF(3,3,$\infty$), allows the separators to have arbitrary coordination. MFCF(4,4,1), is the TMFG (see also DDM).
• Figure 4: A schematic representation of graphical modeling using clique tree representations. Two cliques from separate clique trees $\mathcal{T}_1$ and $\mathcal{T}_2$ are connected by merging the separators $\mathbf{s}_a$ and $\mathbf{s}_b$ into a new separator $\mathbf{s}_c$. The resulting global gain in mutual information is given by $I(\mathbf{s}_a ; \mathbf{s}_b) = H(\mathcal{T}_1) + H(\mathcal{T}_2) - H(\mathcal{T}_3)$. (See also DDM.)
• Figure 5: An HNN layered deep architecture generated from an IFN made of a triangular clique and an attached edge. The HNN is approximating $f(x_1, x_2, x_3, x_4)$ as composite function $h(h_t(h_a(x_1, x_2),h_b(x_2, x_3), h_c(x_1, x_3)),h_d(x_3, x_4))$ where the gathering of the variables in the IFN becomes a gathering into functions of functions. This approach to neural network architecture design was first proposed in wang2023homological.