Diversified and Adaptive Negative Sampling on Knowledge Graphs

TL;DR

A generative adversarial approach called Diversified and Adaptive Negative Sampling DANS on knowledge graphs equipped with a two-way generator that generates more diverse negative triplets through two pathways, and an adaptive mechanism that produces more fine-grained examples by localizing the global generator for different entities and relations.

Abstract

In knowledge graph embedding, aside from positive triplets (ie: facts in the knowledge graph), the negative triplets used for training also have a direct influence on the model performance. In reality, since knowledge graphs are sparse and incomplete, negative triplets often lack explicit labels, and thus they are often obtained from various sampling strategies (eg: randomly replacing an entity in a positive triplet). An ideal sampled negative triplet should be informative enough to help the model train better. However, existing methods often ignore diversity and adaptiveness in their sampling process, which harms the informativeness of negative triplets. As such, we propose a generative adversarial approach called Diversified and Adaptive Negative Sampling DANS on knowledge graphs. DANS is equipped with a two-way generator that generates more diverse negative triplets through two pathways, and an adaptive mechanism that produces more fine-grained examples by localizing the global generator for different entities and relations. On the one hand, the two-way generator increase the overall informativeness with more diverse negative examples; on the other hand, the adaptive mechanism increases the individual sample-wise informativeness with more fine-grained sampling. Finally, we evaluate the performance of DANS on three benchmark knowledge graphs to demonstrate its effectiveness through quantitative and qualitative experiments.

Figures (6)

• Figure 1: Overall framework of DANS. The toy example only shows how to generate fake tail entities, while generating fake head entities follows a similar process.
• Figure 2: Investigation of research questions. Each dataset uses its own $y$-axis, and the metric used on the $y$-axes is MRR. (a) Ablation study on the contribution of individual generator pathways and adaptive FiLM layer. (b) Study of parameter sensitivity on the number of negative triplets $N_s$ and (c) extent of adaptive regularization $\lambda$.
• Figure : (a) $\mathtt{hasPart}$ (WN18RR)
• Figure : (a) $\mathtt{hasPart}$ (WN18RR)
• Figure : (b) $\mathtt{animalType}$ (NELL-995)