Rethinking Regularization Methods for Knowledge Graph Completion

TL;DR

The paper addresses pervasive overfitting in knowledge graph completion by rethinking regularization, introducing SPR, a sparse-regularization method that selectively penalizes dominant embedding coordinates while discarding less informative ones. SPR is designed to be universally applicable across CP/ComplEx-like tensor models, GNN-based approaches, and temporal KGC, and is supported by theoretical sparsification bounds and empirical gains across diverse datasets. Across benchmarks such as WN18RR, FB15K-237, YAGO3-10, and ICEWS, SPR reduces variance and enhances generalization, often raising MRR and Hits@k beyond baseline methods. The work also provides comparisons to dropout and discusses implications for optimizing model complexity, with plans to release code and data for reproducibility and future extensions to adaptive delta and NLP-based KGC systems.

Abstract

Knowledge graph completion (KGC) has attracted considerable attention in recent years because it is critical to improving the quality of knowledge graphs. Researchers have continuously explored various models. However, most previous efforts have neglected to take advantage of regularization from a deeper perspective and therefore have not been used to their full potential. This paper rethinks the application of regularization methods in KGC. Through extensive empirical studies on various KGC models, we find that carefully designed regularization not only alleviates overfitting and reduces variance but also enables these models to break through the upper bounds of their original performance. Furthermore, we introduce a novel sparse-regularization method that embeds the concept of rank-based selective sparsity into the KGC regularizer. The core idea is to selectively penalize those components with significant features in the embedding vector, thus effectively ignoring many components that contribute little and may only represent noise. Various comparative experiments on multiple datasets and multiple models show that the SPR regularization method is better than other regularization methods and can enable the KGC model to further break through the performance margin.

Figures (11)

• Figure 1: Comparison of the overfitting performance of the GIE cao2022geometry model without regularization and with regularization training on the FB15K-237 dataset: Visual comparison of the model Loss, MRR, MR, and Hits@1 indicators as they change with Epoch. The shaded part represents the difference between the training set and the validation set. By adding the regularization method, the variance is significantly reduced.
• Figure 2: The pipeline of the KGC model with/without regularization methods.
• Figure 3: Analysis of the Impact of Regularization Rate on Model Performance for WN18RR, FB15K-237 and YAGO3-10 Datasets.
• Figure 4: Impact of SPR threshold $\delta$ on GIE and ComplEx models across WN18RR, FB15k-237, and YAGO3-10 datasets.
• Figure 5: Comparing the Impact of Regularization on GNN-based KGC models CompGCN: MRR Performance of CompGCN-TransE and CompGCN-ConvE on FB15K-237 and WN18RR Datasets with No-Reg, Dropout, and SPR Settings.