Low-Dimensional Hyperbolic Knowledge Graph Embeddings

TL;DR

<3-5 sentence high-level summary>

Abstract

Knowledge graph (KG) embeddings learn low-dimensional representations of entities and relations to predict missing facts. KGs often exhibit hierarchical and logical patterns which must be preserved in the embedding space. For hierarchical data, hyperbolic embedding methods have shown promise for high-fidelity and parsimonious representations. However, existing hyperbolic embedding methods do not account for the rich logical patterns in KGs. In this work, we introduce a class of hyperbolic KG embedding models that simultaneously capture hierarchical and logical patterns. Our approach combines hyperbolic reflections and rotations with attention to model complex relational patterns. Experimental results on standard KG benchmarks show that our method improves over previous Euclidean- and hyperbolic-based efforts by up to 6.1% in mean reciprocal rank (MRR) in low dimensions. Furthermore, we observe that different geometric transformations capture different types of relations while attention-based transformations generalize to multiple relations. In high dimensions, our approach yields new state-of-the-art MRRs of 49.6% on WN18RR and 57.7% on YAGO3-10.

Figures (8)

• Figure 1: A toy example showing how KGs can simultaneously exhibit hierarchies and logical patterns.
• Figure 2: An illustration of the exponential map $\mathrm{exp}_\mathbf{x}(\mathbf{v})$, which maps the tangent space $\mathcal{T}_\mathbf{x}M$ at the point $\mathbf{x}$ to the hyperbolic manifold $M$.
• Figure 3: Euclidean (left) and hyperbolic (right) isometries. In hyperbolic space, the distance between start and end points after applying rotations or reflections is much larger than the Euclidean distance; it approaches the sum of the distances between the points and the origin, giving more "room" to separate embeddings. This is similar to trees, where the shortest path between two points goes through their nearest common ancestor.
• Figure 4: WN18RR MRR dimension for $d\in\{10, 16, 20, 32, 50, 200, 500\}$. Average and standard deviation computed over 10 runs for RotH.
• Figure 5: (a): RotH offers improved performance in low dimensions; in high dimensions, fixed curvature degrades performance, while trainable curvature approximately recovers Euclidean space. (b): As the dimension increases, the learned curvature of hierarchical relationships tends to zero.