Symmetry-driven embedding of networks in hyperbolic space

TL;DR

This work tackles the lack of uncertainty quantification in hyperbolic network embeddings by introducing BIGUE, a Bayesian MCMC method that samples the posterior of a $ ext{S}^1$ hyperbolic random graph model. By incorporating cluster transformations, BIGUE overcomes multimodal posterior geometry and achieves superior mixing relative to random-walk and gradient-based baselines, enabling credible intervals for vertex positions, model parameters, and derived network properties. The results demonstrate multimodality and non-Gaussian posteriors, and show that posterior samples yield graphs with properties and predictive capabilities comparable to or broader than point-estimate methods like Mercator. The approach provides a principled framework for uncertainty propagation in hyperbolic embeddings with potential applications in edge prediction, routing, and network analysis, while highlighting current scalability limits and avenues for extension to higher dimensions and efficiency enhancements.

Abstract

Hyperbolic models are known to produce networks with properties observed empirically in most network datasets, including heavy-tailed degree distribution, high clustering, and hierarchical structures. As a result, several embeddings algorithms have been proposed to invert these models and assign hyperbolic coordinates to network data. Current algorithms for finding these coordinates, however, do not quantify uncertainty in the inferred coordinates. We present BIGUE, a Markov chain Monte Carlo (MCMC) algorithm that samples the posterior distribution of a Bayesian hyperbolic random graph model. We show that the samples are consistent with current algorithms while providing added credible intervals for the coordinates and all network properties. We also show that some networks admit two or more plausible embeddings, a feature that an optimization algorithm can easily overlook.

Figures (7)

• Figure 1: Probabilistic hyperbolic embedding of a synthetic graph with BIGUE. Hyperbolic coordinates are obtained using a transformation between the $\mathbb{H}^2$ and the $\mathbb{S}^1$ models described in Supplementary Note 2. Black points and dark-colored points are the median coordinates of each vertex. Light-colored points are the $2000$ sampled positions for the three highlighted vertices. Lines are edges drawn using hyperbolic geodesics. The synthetic graph of 30 vertices is generated with the $\mathbb{S}^1$ likelihood, with angular coordinates drawn from their prior, $\beta=2.5$ and the $\kappa$ parameters drawn from a Pareto distribution of exponent $-2.5$ over the truncated interval $(4, 10)$.
• Figure 2: Log-likelihood of the model when a single vertex is moved along the circle. Each dip is a divergence that occurs when a vertex is positioned at the same position as a disconnected vertex. These divergences form barriers in the objective function landscape (log-likelihood, posterior distribution), which is one of the reasons why hyperbolic embedding is a difficult problem. The likelihood is computed on the ground truth embedding of the synthetic graph of Fig. \ref{['fig:probabilistic_embedding']}. Colors indicate the degree of the moved vertex (lowest degree in light blue, median degree in blue and highest degree in dark blue).
• Figure 3: Statistics of Markov chain Monte Carlo algorithms when embedding the synthetic graph of Fig. \ref{['fig:probabilistic_embedding']}. Results for the random walk algorithm (RW) and for our method (BIGUE) that uses both cluster transformations and a random walk (RW+CT) are displayed in red and blue, respectively. The orange curves show the results for dynamic Hamiltonian Monte Carlo (HMC) for the differentiable $\mathbb{S}^1$ model described in Supplementary Note 3. The green line displays the maximum log-likelihood obtained from Mercator after $10$ runs, each with $10$ refinements garcia-perez_2019_MercatorUncoveringFaithful and the black line is the ground truth embedding's log-likelihood. (a) Normalized autocovariance averaged over all parameters and chains at different lags. (b-c) Traceplot of log-likelihood of a simulated Markov chain initialized without (panel b) and with (panel c) access to ground truth information. When the ground truth is unknown, we initialize $\kappa_u=\deg(u)$ and draw the other parameters from their priors. (d) Traceplot of log-likelihood of a Markov chain after thinning the chain shown in panel b. The blue line of this panel is the sample shown in Fig. \ref{['fig:probabilistic_embedding']}. In each case, we compute $\hat{R}_\text{max}$, the maximum potential scale reduction factor for all parameters after the iterations removed from the warm-up (grey part of traceplots)---values close to 1 are desirable. Four independent chains were simulated to compute $\hat{R}$ (shown on each panel with the color corresponding to the sampling algorithm) and the autocovariance, but only one is displayed (representative of the four).
• Figure 4: Summary of the cluster transformations used in BIGUE. The flip transformation targets the reflection symmetries, and the exchange and translate transformations target the rotation symmetries.
• Figure 5: Posterior estimates and posterior predictive distribution for the synthetic graph used in Fig. \ref{['fig:alg_comparison_30v']}. (a) Angular coordinates $\theta$. (b) Parameters $\kappa$. (c) Inverse temperature $\beta$. (d) Density. (e) Clustering (the transitivity). (f) Greedy routing success rate. (g) Global hierarchy level garcia-perez_2016_HiddenHyperbolicGeometry. (h) Normalized rank of the removed edges' existence probability among all the unconnected pairs (see below). In each case, $2000$ total embeddings were sampled from four independent chains. All shades of blue, green and gray display the values for the posterior sample, Mercator and the ground truth, respectively. In panels (a,b), points denote the median, and the error bars cover the interquartile range. In panels (c-g), the vertical dotted lines show point estimates computed with the Mercator and ground truth embeddings. In panels (d,e), we generated graph samples for Mercator and the ground truth by conditioning the model's likelihood on point estimates of the embedding. In panels (c, f,g), the blue dashed line is the median of the posterior sample. In panel (h), we test the algorithm on a link prediction task kumar_2020_LinkPredictionTechniques, and report the rank of removed edges as predicted by the likelihood. Pairs of embedding and graphs of 30 vertices were sampled from the prior. For each graph, $5\%$ of the edges were then randomly removed ($4$ to $5$ edges), and the normalized ranks of removed edges were calculated and binned. (Disconnected graphs were rejected for compatibility with Mercator). The figure reports the median and interquartile range of the normalized rank frequencies calculated across graphs, removals, and samples. The AUC values are shown in Supplementary Note 5.