Understanding Expressivity of GNN in Rule Learning

TL;DR

The paper addresses how expressive modern GNNs are for learning logical rules in knowledge graphs by unifying tail-entity–scoring GNNs under the QL-GNN framework and analyzing their rule-learning capacity through graded modal logic. It shows QL-GNN can learn rule formulas in the class $ ext{CML}[G,\mathsf{h}]$, with examples like chain-like and inductive rules, and provides a Corollary for constructive rule formation. To overcome limitations, it introduces EL-GNN, which labels additional entities using a degree-based scheme to capture more complex rule structures, at linear time cost, and demonstrates improved expressivity. Empirical results on synthetic and real KG datasets validate the theory and illustrate that EL-GNN often outperforms strong baselines (NBFNet, RED-GNN), providing a principled approach to designing labeling strategies for rule learning in KG reasoning.

Abstract

Rule learning is critical to improving knowledge graph (KG) reasoning due to their ability to provide logical and interpretable explanations. Recently, Graph Neural Networks (GNNs) with tail entity scoring achieve the state-of-the-art performance on KG reasoning. However, the theoretical understandings for these GNNs are either lacking or focusing on single-relational graphs, leaving what the kind of rules these GNNs can learn an open problem. We propose to fill the above gap in this paper. Specifically, GNNs with tail entity scoring are unified into a common framework. Then, we analyze their expressivity by formally describing the rule structures they can learn and theoretically demonstrating their superiority. These results further inspire us to propose a novel labeling strategy to learn more rules in KG reasoning. Experimental results are consistent with our theoretical findings and verify the effectiveness of our proposed method. The code is publicly available at https://github.com/LARS-research/Rule-learning-expressivity.

Figures (6)

• Figure 1: The existence of a triplet in KG is determined by the corresponding rule structure. We investigates the kind of rule structures can be learned by SOTA GNNs for KG reasoning (i.e., QL-GNN), and proposes EL-GNN, which can learn more rule structures compared to QL-GNN.
• Figure 2: Example of rule structures and their corresponding rule formulas QL-GNN can learn.
• Figure 3: Two rule structures cannot be distinguished by QL-GNN.
• Figure 4: Accuracy versus out-degree $d$ of EL-GNN on the dataset with relation $U$.
• Figure 5: Some rule structures in real datasets. The rule structure (a) is from Family dataset and is not a rule formula in $\text{CML}[G, \mathsf{h}]$, which cannot not be learned by QL-GNN. The rule structures (b) and (c) are from FB15k-237 dataset and are rule formulas in $\text{CML}[G, \mathsf{h}]$, which can be learned by QL-GNN.

Theorems & Definitions (47)

• Proposition 3.1
• Theorem 3.2: Logical expressivity of QL-GNN
• Corollary 3.3
• Theorem 3.4: Logical expressivity of CompGCN
• Remark
• Proposition 4.1
• Corollary 4.2
• proof : proof of Proposition \ref{['prop:QL']}
• Definition A.1: Definition of graded modal logic
• Corollary A.2