Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MQuinE: a cure for "Z-paradox" in knowledge graph embedding models

Yang Liu, Huang Fang, Yunfeng Cai, Mingming Sun

TL;DR

A new KGE model called MQuinE is proposed that does not suffer from Z-paradox while preserves strong expressiveness to model various relation patterns including symmetric/asymmetric, inverse, 1-N/N-1/N-N, and composition relations with theoretical justification.

Abstract

Knowledge graph embedding (KGE) models achieved state-of-the-art results on many knowledge graph tasks including link prediction and information retrieval. Despite the superior performance of KGE models in practice, we discover a deficiency in the expressiveness of some popular existing KGE models called \emph{Z-paradox}. Motivated by the existence of Z-paradox, we propose a new KGE model called \emph{MQuinE} that does not suffer from Z-paradox while preserves strong expressiveness to model various relation patterns including symmetric/asymmetric, inverse, 1-N/N-1/N-N, and composition relations with theoretical justification. Experiments on real-world knowledge bases indicate that Z-paradox indeed degrades the performance of existing KGE models, and can cause more than 20\% accuracy drop on some challenging test samples. Our experiments further demonstrate that MQuinE can mitigate the negative impact of Z-paradox and outperform existing KGE models by a visible margin on link prediction tasks.

MQuinE: a cure for "Z-paradox" in knowledge graph embedding models

TL;DR

A new KGE model called MQuinE is proposed that does not suffer from Z-paradox while preserves strong expressiveness to model various relation patterns including symmetric/asymmetric, inverse, 1-N/N-1/N-N, and composition relations with theoretical justification.

Abstract

Knowledge graph embedding (KGE) models achieved state-of-the-art results on many knowledge graph tasks including link prediction and information retrieval. Despite the superior performance of KGE models in practice, we discover a deficiency in the expressiveness of some popular existing KGE models called \emph{Z-paradox}. Motivated by the existence of Z-paradox, we propose a new KGE model called \emph{MQuinE} that does not suffer from Z-paradox while preserves strong expressiveness to model various relation patterns including symmetric/asymmetric, inverse, 1-N/N-1/N-N, and composition relations with theoretical justification. Experiments on real-world knowledge bases indicate that Z-paradox indeed degrades the performance of existing KGE models, and can cause more than 20\% accuracy drop on some challenging test samples. Our experiments further demonstrate that MQuinE can mitigate the negative impact of Z-paradox and outperform existing KGE models by a visible margin on link prediction tasks.
Paper Structure (29 sections, 7 theorems, 29 equations, 3 figures, 13 tables, 1 algorithm)

This paper contains 29 sections, 7 theorems, 29 equations, 3 figures, 13 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related works
  3. Z-paradox and its cure: MQuinE
  4. Z-paradox
  5. MQuinE
  6. Expressiveness of MQuinE
  7. Learning of MQuinE
  8. Parameterization and regularization.
  9. Z-sampling.
  10. Objective function.
  11. Experiments
  12. Experimental setup
  13. Statistics of Z-patterns
  14. Case study of MQuinE
  15. Overall evaluation on link prediction
  16. ...and 14 more sections

Key Result

Proposition 3.1

Given a KGE model parameterized by $\{ {\bm{e}}_i \}_{i=1}^{|\mathcal{E}|}, \{ {\bm{r}}_i \}_{i=1}^{|\mathcal{R}|}$ and a score function $s(\cdot)$. If $s( {\bm{h}}, {\bm{r}}, {\bm{t}} )$ can be expressed as $\| f( {\bm{h}}, {\bm{r}} ) - g( {\bm{t}}, {\bm{r}} ) \|$ for some functions $f(\cdot)$ and

Figures (3)

  • Figure 1.1: Left panel: an illustration of Z-paradox. Right panel: an illustration of Z-paradox in the YAGO3-10 dataset.
  • Figure 5.1: Case splitting description.
  • Figure 5.2: Statistics of Z-patterns in the testing set responding to the training set.

Theorems & Definitions (19)

  • Definition 1: Z-paradox
  • Proposition 3.1
  • proof
  • Remark 3.1
  • Remark 3.2
  • Theorem 3.2
  • Theorem 3.3: Composition
  • Theorem 3.4: No Z-paradox
  • proof
  • Remark 3.3
  • ...and 9 more