Shedding Light on Problems with Hyperbolic Graph Learning

TL;DR

The paper challenges the claimed advantages of hyperbolic graph learning by showing that a carefully trained Euclidean baseline can match or surpass hyperbolic models on standard tasks, even for datasets labeled as highly hyperbolic by $\delta$-hyperbolicity. It identifies three core problems—poor Euclidean baselines, tasks that poorly discriminate geometry, and coarse geometry metrics—responsible for inflated hyperbolic performance. The authors propose concrete remedies, including a well-tuned Euclidean baseline, synthetic Tree benchmarks with controlled graph-feature interactions, and richer dataset geometry characterizations. They introduce a parametric family Tree$(b,\ell,\gamma,\delta,\mathcal{D})$ and a concrete Tree1111 dataset to stress-test geometry-dependent learning. Overall, the work urges rigorous baselines and geometry-aware benchmarking to accurately assess when hyperbolic representations offer real benefits for graph tasks.

Abstract

Recent papers in the graph machine learning literature have introduced a number of approaches for hyperbolic representation learning. The asserted benefits are improved performance on a variety of graph tasks, node classification and link prediction included. Claims have also been made about the geometric suitability of particular hierarchical graph datasets to representation in hyperbolic space. Despite these claims, our work makes a surprising discovery: when simple Euclidean models with comparable numbers of parameters are properly trained in the same environment, in most cases, they perform as well, if not better, than all introduced hyperbolic graph representation learning models, even on graph datasets previously claimed to be the most hyperbolic as measured by Gromov $δ$-hyperbolicity (i.e., perfect trees). This observation gives rise to a simple question: how can this be? We answer this question by taking a careful look at the field of hyperbolic graph representation learning as it stands today, and find that a number of results do not diligently present baselines, make faulty modelling assumptions when constructing algorithms, and use misleading metrics to quantify geometry of graph datasets. We take a closer look at each of these three problems, elucidate the issues, perform an analysis of methods, and introduce a parametric family of benchmark datasets to ascertain the applicability of (hyperbolic) graph neural networks.

Figures (4)

• Figure 1: Above is an example where we give the best Euclidean embedding of a tripodal graph, associated with a toy graph dataset (comprised of the tripodal tree and associated node features, $f_i$), in $\mathbb{R}^2$, i.e., the embedding that has lowest average distortion. We see that the largest amount of separation we can get between the leaf nodes while maintaining the distances between the root and leaf nodes is $\sqrt{3} \approx 1.7$, falling short of $2$. In contrast, we give the best hyperbolic embedding of the same tree in $\mathbb{H}^2$. Note that the embedding has high geometric fidelity, i.e., all distances can be preserved. We compute this embedding explicitly in Appendix \ref{['sec:hypexample']}.
• Figure 2: Introducing parental dependence in node features very quickly leads the MLP to solve link prediction. Each data point is obtained by tuning the MLP on the relevant synthetic dataset. The average of $5$ trials is reported and the error region specifies one standard deviation.
• Figure 3: An illustration of the $\delta$-slim triangle condition for the metric space $(X,d)$. Recall $B_\delta(S) := \{p | \inf_{s \in S}d(s, p) < \delta\}$.
• Figure 4: Ollivier-Ricci curvature Lin2011RicciCO for two graphs. The left graph is the Disease chami2019hyperbolic dataset and the right graph is a tree with a branch factor of $10$ from the synthetic graph dataset Tree1111 we introduce.