Neural Embeddings of Graphs in Hyperbolic Space

TL;DR

The paper argues that Euclidean vector spaces are ill-suited for embedding complex networks with hierarchical, power-law structure. It proposes neural graph embeddings in hyperbolic space using two Poincaré disks and a hyperbolic Skipgram framework with negative sampling, deriving backpropagation in hyperbolic coordinates. Empirical results on five public networks show significant improvements in vertex attribution over Euclidean DeepWalk, with visualizations illustrating clearer community separation in hyperbolic space. The work demonstrates that exploiting hyperbolic geometry can yield compact, discriminative representations that better reflect the natural organization of complex networks.

Abstract

Neural embeddings have been used with great success in Natural Language Processing (NLP). They provide compact representations that encapsulate word similarity and attain state-of-the-art performance in a range of linguistic tasks. The success of neural embeddings has prompted significant amounts of research into applications in domains other than language. One such domain is graph-structured data, where embeddings of vertices can be learned that encapsulate vertex similarity and improve performance on tasks including edge prediction and vertex labelling. For both NLP and graph based tasks, embeddings have been learned in high-dimensional Euclidean spaces. However, recent work has shown that the appropriate isometric space for embedding complex networks is not the flat Euclidean space, but negatively curved, hyperbolic space. We present a new concept that exploits these recent insights and propose learning neural embeddings of graphs in hyperbolic space. We provide experimental evidence that embedding graphs in their natural geometry significantly improves performance on downstream tasks for several real-world public datasets.

Figures (5)

• Figure 1: Illustrations of properties of hyperbolic space. \ref{['fig:circle_limit1']} Tiles of constant area \ref{['fig:parallel']} Parallel lines.
• Figure 2: Geometric interpretation of the update equations in the Skipgram model. The vector representation of the output vertex $v_{w_O}^{\prime(\mathrm{new})}$ is moved closer (blue) to the vector representation of the input vertex $v_I$, while all other vectors $v_{w_j}^{\prime(\mathrm{new})}$ move further away (red). The magnitude of the change is proportional to the prediction error.
• Figure 3: The skipgram model predicts the context vertices from a single input vertex.
• Figure 4: The factions of the Zachary karate network are easily linearly separable when embedded in 2D hyperbolic space. This is not true when embedding in Euclidean space. Both embeddings were run for 5 epochs on the same random walks
• Figure 5: Macro F1 score ($y$-axis) against percentage of labelled vertices used for training ($x$-axis). In all cases hyperbolic embeddings (blue) significantly outperform Euclidean deepwalk embeddings (red). Error bars show standard error from the mean over ten repetitions. The legend used in subfigure (a) applies to all subfigures. A consistent trend across the datasets is that an embedding into a 2D hyperbolic space outperforms deepwalk architectures with embeddings ranging from 2D to 128D.