LinkLogic: A New Method and Benchmark for Explainable Knowledge Graph Predictions

TL;DR

LinkLogic addresses the interpretability gap in knowledge graph link prediction by surfacing and ranking explanation paths through perturbation-based features, fitted by a non-negative surrogate to explain the KGE score. Using FB13 and FB14 family structures, the authors construct the Parents Benchmark to enable rigorous evaluation of explanation fidelity ($R^2$), selectivity (path count), and relevance ($NDCG@k$). Empirical results show higher fidelity for true facts, controlled explanation complexity, and meaningful qualitative insights from family-tree experiments (tautologies, removal of direct links, and sibling vocabulary). The work establishes a principled evaluation framework and a reproducible explanation method that can guide future benchmarking of KGE explainability approaches and motivate extensions to negative explanations and more expressive surrogates.

Abstract

While there are a plethora of methods for link prediction in knowledge graphs, state-of-the-art approaches are often black box, obfuscating model reasoning and thereby limiting the ability of users to make informed decisions about model predictions. Recently, methods have emerged to generate prediction explanations for Knowledge Graph Embedding models, a widely-used class of methods for link prediction. The question then becomes, how well do these explanation systems work? To date this has generally been addressed anecdotally, or through time-consuming user research. In this work, we present an in-depth exploration of a simple link prediction explanation method we call LinkLogic, that surfaces and ranks explanatory information used for the prediction. Importantly, we construct the first-ever link prediction explanation benchmark, based on family structures present in the FB13 dataset. We demonstrate the use of this benchmark as a rich evaluation sandbox, probing LinkLogic quantitatively and qualitatively to assess the fidelity, selectivity and relevance of the generated explanations. We hope our work paves the way for more holistic and empirical assessment of knowledge graph prediction explanation methods in the future.

Figures (9)

• Figure 1: Directed sub-graph representing the Mozart family. Black and red edges are included in FB13. The purple sibling edge highlighted was included in FB14. The link in red shows the query triple, (Maria, parent, Leopold).
• Figure 2: Quantitative analysis of LinkLogic explanation relevance, selectivity, and fidelity. A KGE prediction scores, B number of paths per explanation, and C LinkLogic fidelity, for 1300 explanations generated for Nonsense, False, and True query triples across the 3 relation categories in FB13. D Relevance assessment across the full Parents benchmark, E Number of paths per explanation as a function of the number of siblings, and F LinkLogic fidelity versus the number of siblings present in the family. For the path score heuristic, 0.90 and 0.95 correspond to the KGE score threshold at which paths were selected.
• Figure 3: Most frequently occurring paths in LinkLogic prediction explanations for parent relationships, based on various configurations: A before and after removing the direct child link, and B before and after adding the sibling relation to training. $p$=parent, $c$=child, $s$=sibling, $p_2$=co-parent, $x$=any entity that is not $p, c, s,$or $p_2$.
• Figure 4: A Scatter plot of real and imaginary parts of the FB13 spouse relation embedding against FB13 children relation embedding , B Scatter plot of only the real parts of the FB14 spouse (green) and sibling (yellow) relation embedding against the FB14 children relation. C Scatter plots of real and imaginary parts of the FB14 sibling embeddings versus the FB14 spouse embeddings. D The change in coefficient of those spouse links (FB13) that were correctly identified as siblings (FB14).
• Figure A.1.1: Distribution of different relation types in the FB13 and FB14 dataset