A Mechanistic Study on the Impact of Entity Degree Distribution in Open-World Link Prediction
Jiang Xiaobo, Yongru Chen
TL;DR
The paper addresses the bottleneck in open-world link prediction by focusing on entity degree distribution as a key structural factor. It combines Sobol sensitivity analysis, correlation analyses, and a two-stage mechanistic examination of training dynamics to show that higher-degree entities are learned more effectively, leading to better overall prediction. By introducing an embedding quality index $Q$ and analyzing both the KGE pretraining and the text-to-embedding mapping stages, the study reveals how degree distribution shapes embedding space and gradient contributions, explaining performance gaps. The findings offer practical optimization directions, including degree-aware sampling, data augmentation, and adaptive loss functions, to mitigate degree bias and improve OW-LP performance in real-world knowledge graphs.
Abstract
Open-world link prediction supports the knowledge representation and link prediction of new entities, enhancing the practical value of knowledge graphs in real-world applications. However, as research deepens, the performance improvements in open-world link prediction have gradually reached a bottleneck. Understanding its intrinsic impact mechanisms is crucial for identifying the key factors that limit performance, offering new theoretical insights and optimization strategies to overcome these bottlenecks. This study focuses on entity degree distribution, a core structural feature of knowledge graphs, and investigates its impact on the performance of open-world link prediction tasks. First, through experimental analysis, we confirm that entity degree distribution significantly affects link prediction model performance. Second, we reveal a strong positive correlation between entity degree and link prediction accuracy. Moreover, this study explores how entity degree influences embedding space distribution and weight updates during neural network training, uncovering the deeper mechanisms affecting open-world link prediction performance. The findings show that entity degree distribution has a significant impact on model training. By influencing the quality of the embedding space and weight updates, it indirectly affects the overall prediction performance of the model. In summary, this study not only highlights the critical role of entity degree distribution in open-world link prediction but also uncovers the intrinsic mechanisms through which it impacts model performance, providing valuable insights and directions for future research in this field.