Flexible inference in heterogeneous and attributed multilayer networks

TL;DR

PIHAM introduces a flexible Bayesian generative model for attributed multilayer networks that can ingest heterogeneous data types across arbitrary layers. By using Gaussian priors/posteriors and Laplace Matching with automatic differentiation, it yields interpretable latent mixed memberships and maps posteriors to domain-constrained distributions (e.g., Dirichlet) for clear interpretation. The method demonstrates strong prediction and community-detection performance on synthetic heterogeneous data and a real-world social network from rural India, while providing principled posterior summaries to interpret complex mixtures of information. This approach enables scalable, black-box inference for complex networks with diverse node attributes and edge types, advancing practical analysis of real-world multilayer systems.

Abstract

Networked datasets can be enriched by different types of information about individual nodes or edges. However, most existing methods for analyzing such datasets struggle to handle the complexity of heterogeneous data, often requiring substantial model-specific analysis. In this paper, we develop a probabilistic generative model to perform inference in multilayer networks with arbitrary types of information. Our approach employs a Bayesian framework combined with the Laplace matching technique to ease interpretation of inferred parameters. Furthermore, the algorithmic implementation relies on automatic differentiation, avoiding the need for explicit derivations. This makes our model scalable and flexible to adapt to any combination of input data. We demonstrate the effectiveness of our method in detecting overlapping community structures and performing various prediction tasks on heterogeneous multilayer data, where nodes and edges have different types of attributes. Additionally, we showcase its ability to unveil a variety of patterns in a social support network among villagers in rural India by effectively utilizing all input information in a meaningful way.

Figures (10)

• Figure 1: Input data and graphical model representation. (A) The attributed multilayer network is represented by the interactions $A_{ij}^{\ell}$ and the node attributes $X_{ix}$. (B) PIHAM describes the observed data through a set of latent variables $\boldsymbol{\Theta} = (\boldsymbol{U}, \boldsymbol{V}, \boldsymbol{W}, \boldsymbol{H})$. $\boldsymbol{U_i}$ and $\boldsymbol{V_i}$ respectively depict the communities of node $i$ determined by the out-going and in-coming edges; $\boldsymbol{W^{\ell}}$ is the affinity matrix associated to the layer $\ell$ and characterizes the edge density between different community pairs in the given layer; $\boldsymbol{H_{\cdot x}}$ is a $K$-dimensional vector that explains how an attribute $x$ is distributed among the $K$ communities. All latent variables are independent and normally distributed, and $f(\cdot)$ and $g(\cdot)$ are transformation functions to ensure that the expected values $\lambda_{ij}^{\ell}$ and $\pi_{ix}$ belong to the correct parameter space for the various distribution types.
• Figure 2: Prediction performance on synthetic data. We analyze synthetic attributed multilayer networks with $L=3$ heterogeneous layers (one with binary interactions (A), one with nonnegative discrete weights (B), and one with real values(C)), three node covariates (one categorical with $Z=4$ categories (D), one representing nonnegative discrete values (E), and one involving real values (F)), varying number of nodes $N$, and diverse number of overlapping communities $K$. We employ a $5$-fold cross-validation procedure and plot averages and confidence intervals over $20$ independent samples. The prediction performances are measured with different metrics according to the data type: Area Under the receiver-operator Curve (AUC) for binary interactions (A), the Maximum Absolute Error (MAE) for nonnegative discrete values (B, E), the Root Mean Squared Error (RMSE) for real values (C, F), and accuracy for categorical attributes (D). The baselines are given by the predictions obtained from either the average or the maximum frequency in the training set. For the categorical attribute, we also include the uniform random probability over $Z$, and for the AUC, the baseline corresponds to the random choice $0.5$. Overall, PIHAM outperforms the baselines significantly for each type of information.
• Figure 3: Interpretation of posterior distributions in comparison with ground truth memberships. We analyze a synthetic attributed multilayer network with ground truth mixed-memberships represented as normalized vectors summing to $1$. In this case, $K=3$. (Top row) Ground truth membership vectors for three representative nodes: Node A displays extreme mixed-membership, Node B shows a slightly lower mixed-membership, and Node C exhibits hard-membership. (Middle row) Inferred posterior distributions $\hat{U}_{ik} \sim \mathcal{N}(\hat{U}_{ik}; \hat{\mu}^{U}_{ik}, (\hat{\sigma}^{U}_{ik})^{2})$, where different colors represent distinct communities, and the distribution in gray consists of the $L_2$-barycenter distribution. Overlap is the average of the area of overlap between every pair of distributions, and $\sigma^2$ is the variance of the barycenter distribution. (Bottom row) Transformed posterior distributions into the simplex space using the LM technique and employing Dirichlet distributions. The inferred node memberships reflect the ground truth behavior, as evidenced by the trends of Overlap and $\sigma^2$, which align with the decreasing degree of true mixed-membership. Additionally, the Dirichlet transformation provides a more straightforward interpretation, further supporting this conclusion.
• Figure 4: Inference of overlapping communities in a social support network. We analyze a real-world heterogeneous attributed multilayer network, which was collected in 2013 through surveys in the Indian village. This network comprises six binary layers representing directed social support interactions among individuals, alongside an additional layer reflecting information proportional to the distance between individuals’ households. (Top row) As node covariates, we consider caste $\boldsymbol{X_{\cdot 1}}$, religion $\boldsymbol{X_{\cdot 2}}$, and years of education $\boldsymbol{X_{\cdot 3}}$. For privacy reasons, nodes belonging to castes with fewer than five individuals are aggregated into an "Other" category. Moreover, the displayed interactions refer only to the first layer (talk about important matters) to enhance clarity in visualization. (Middle-Bottom rows) We display the MAP estimates of the out-going communities inferred by PIHAM. For easier interpretation, we apply a $\mathop{\mathrm{softmax}}\nolimits$ transformation to the MAP estimates of the membership vectors, and darker values in the grayscale indicate higher values in the membership vector $\hat{\boldsymbol{U}}_{\boldsymbol{i}}$. The position of the nodes reflects the geographical distance between individuals' households. In summary, the inferred communities do not exclusively align with a single type of information. Rather, PIHAM incorporates all input information to infer partitions that effectively integrate them in a meaningful way.
• Figure S5: Prediction and community detection performance on synthetic data. We analyze synthetic attributed multilayer networks with $N=500$ nodes, $L=2$ layers (one being assortative and the other disassortative), a categorical attribute with $Z=3$ categories, $K=3$ overlapping communities, and increasing average degrees $\langle{k}\rangle \in \{10, 15, 20, \dots, 50\}$. The results represent averages and confidence intervals over $20$ independent samples. For prediction tasks, we employ a $5$-fold cross-validation procedure. The evaluation metrics include (A) the AUC for edge prediction, with a baseline of $0.5$ corresponding to random choice, and (B) accuracy for covariate prediction. Here, MRF represents a baseline given by the predictions obtained from the maximum frequency in the training set, while Random denotes the uniform random probability over $Z$. (C) Community detection performance is assessed using Cosine Similarity (CS). As inferred point estimates, we consider both the mean of transformed Dirichlet posterior distributions and the $\mathop{\mathrm{softmax}}\nolimits$ transformation of $\hat{\boldsymbol{\mu}}^{\boldsymbol{\theta}}$. Overall, PIHAM exhibits comparable performance to MTCOV across all tasks despite its broader framework, especially in scenarios involving denser networks.