Block-Diagonal Orthogonal Relation and Matrix Entity for Knowledge Graph Embedding

TL;DR

This work introduces OrthogonalE, a novel KGE model employing matrices for entities and block-diagonal orthogonal matrices with Riemannian optimization for relations that significantly outperforms state-of-the-art KGE models while substantially reducing the number of relation parameters.

Abstract

The primary aim of Knowledge Graph embeddings (KGE) is to learn low-dimensional representations of entities and relations for predicting missing facts. While rotation-based methods like RotatE and QuatE perform well in KGE, they face two challenges: limited model flexibility requiring proportional increases in relation size with entity dimension, and difficulties in generalizing the model for higher-dimensional rotations. To address these issues, we introduce OrthogonalE, a novel KGE model employing matrices for entities and block-diagonal orthogonal matrices with Riemannian optimization for relations. This approach enhances the generality and flexibility of KGE models. The experimental results indicate that our new KGE model, OrthogonalE, is both general and flexible, significantly outperforming state-of-the-art KGE models while substantially reducing the number of relation parameters.

Figures (12)

• Figure 1: Fundamental operations ($\mathbf{e}_{R}^{1} \cdot \mathbf{e}_{h}^{1} \approx \mathbf{e}_{t}^{1}$) and inherent challenges of rotation-based KGE models. Rotation-based methods require increasing relation parameters for adequate entity representation (lack of flexibility) and struggle with researching higher-dimensional rotation embeddings ($d>3$) due to their complexity (lack of generality). OrthogonalE, depicted in Fig. \ref{['fig:approach_OrthogonalE']}, efficiently resolves these challenges.
• Figure 2: Diagram of the OrthogonalE approach. We employ matrices for entities and block-diagonal orthogonal matrices with Riemannian optimization for relations, thereby retaining the advantages of rotation-based method relation patterns while addressing its two main issues.
• Figure 3: Illustration of Riemannian gradient descent iteration on an orthogonal manifold.
• Figure 4: MRR accuracy comparison of OrthogonalE models with different block-diagonal orthogonal matrices across varying entity dimensions ($n\times 1$, where we set $m=1$ to control the entity shape as a single entity vector) on WN18RR and FB15K-237.
• Figure 5: MRR accuracy comparison of OrthogonalE(2$\times$2) and Gram-Schmidt(2$\times$2) models across varying entity dimensions ($m$) with fixed relation matrix (40$\times$40) on WN18RR.