RulE: Knowledge Graph Reasoning with Rule Embedding

TL;DR

RulE presents a unified neural-symbolic framework that jointly embeds entities, relations, and first-order rules in a complex space, enabling soft rule reasoning through learned rule confidences. By integrating a RotatE-style KGE with a rule embedding loss and a soft grounding mechanism, RulE regularizes embeddings and provides a probabilistic rule-based grounding via an MLP over a soft multi-hot encoding. Empirically, RulE improves link prediction across six benchmarks, with particularly strong gains on rule-inferrable graphs, and ablations show the benefit of soft rule reasoning and rule embedding as a bridge between embedding-based and rule-based approaches. The approach offers improved interpretability and flexibility by producing rule confidences and enabling soft rule integration, with potential limitations in scalability due to path enumeration and a focus on chain rules. Overall, RulE advances neural-symbolic KG reasoning by harmonizing logical rules with embedding learning and enabling principled, soft rule inference.

Abstract

Knowledge graph (KG) reasoning is an important problem for knowledge graphs. In this paper, we propose a novel and principled framework called \textbf{RulE} (stands for {Rul}e {E}mbedding) to effectively leverage logical rules to enhance KG reasoning. Unlike knowledge graph embedding (KGE) methods, RulE learns rule embeddings from existing triplets and first-order {rules} by jointly representing \textbf{entities}, \textbf{relations} and \textbf{logical rules} in a unified embedding space. Based on the learned rule embeddings, a confidence score can be calculated for each rule, reflecting its consistency with the observed triplets. This allows us to perform logical rule inference in a soft way, thus alleviating the brittleness of logic. On the other hand, RulE injects prior logical rule information into the embedding space, enriching and regularizing the entity/relation embeddings. This makes KGE alone perform better too. RulE is conceptually simple and empirically effective. We conduct extensive experiments to verify each component of RulE. Results on multiple benchmarks reveal that our model outperforms the majority of existing embedding-based and rule-based approaches.

Figures (3)

• Figure 1: (a) Traditional KGE methods embed entities and relations as low-dimensional vectors only using existing triplets by defining operations between entities and relations (e.g., translation); (b) Our RulE associates each rule with an embedding and additionally defines mathematical operations between relations and logical rules (e.g., multi-step translation) to leverage first-order logical rules.
• Figure 2: Architecture of RulE. It consists of three components. 1) We first model the relationship between entities and relations as well as the relationship between relations and logical rules to learn joint entity, relation and rule embedding in the same continuous space. With the learned rule embeddings ($\bm R$) and relation embeddings ($\bm r$), RulE can output a weight ($w$) as the confidence score of each rule. 2) In the soft rule reasoning stage, we construct a soft multi-hot encoding $\bm v$ based on rule confidences. Specifically, for triplet $(e_1, r_3, e_6)$, only $R_1$ and $R_3$ can infer the fact with the grounding paths $e_1 \rightarrow r_1 \rightarrow r_2 \rightarrow e_6$ and $e_1 \rightarrow r_7 \rightarrow r_8 \rightarrow e_6$ (highlighted with purple and blue). Thus, the value of $\bm v_1$ is $w_1$, $\bm v_3$ is $w_3$ and others (unactivated rules) are $0$. Then the constructed soft multi-hot encoding passes an MLP to output the grounding rule score. 3) Finally, RulE integrates the KGE score calculated from the entity and relation embeddings trained in the first stage and the grounding rule score obtained in the second stage to reason unknown triplets.
• Figure 3: (a) and (b) show the MRR results of RulE with varying $\beta$ on FB15k-237 and WN18RR.