Improving Multi-hop Logical Reasoning in Knowledge Graphs with Context-Aware Query Representation Learning

TL;DR

This work tackles the problem of inaccurate or unstable multi-hop logical reasoning in knowledge graphs by introducing CaQR, a model-agnostic approach that enriches query embeddings with two complementary contexts: structural context from the query graph and relation-induced context from the KG. By incorporating position, role, and type embeddings alongside relation-derived signals, CaQR refines node representations at each projection step, reducing cascading errors in complex queries. Empirical results on FB15k-237 and NELL995 show consistent improvements across Q2B, BetaE, and ConE, with up to 19.5% gains on challenging queries and robust performance across ablations and hyperparameters. This method enhances the interpretability and accuracy of FOL query answering in KGs and can be readily integrated into existing reasoning pipelines to improve multi-hop query understanding and retrieval performance.

Abstract

Multi-hop logical reasoning on knowledge graphs is a pivotal task in natural language processing, with numerous approaches aiming to answer First-Order Logic (FOL) queries. Recent geometry (e.g., box, cone) and probability (e.g., beta distribution)-based methodologies have effectively addressed complex FOL queries. However, a common challenge across these methods lies in determining accurate geometric bounds or probability parameters for these queries. The challenge arises because existing methods rely on linear sequential operations within their computation graphs, overlooking the logical structure of the query and the relation-induced information that can be gleaned from the relations of the query, which we call the context of the query. To address the problem, we propose a model-agnostic methodology that enhances the effectiveness of existing multi-hop logical reasoning approaches by fully integrating the context of the FOL query graph. Our approach distinctively discerns (1) the structural context inherent to the query structure and (2) the relation-induced context unique to each node in the query graph as delineated in the corresponding knowledge graph. This dual-context paradigm helps nodes within a query graph attain refined internal representations throughout the multi-hop reasoning steps. Through experiments on two datasets, our method consistently enhances the three multi-hop reasoning foundation models, achieving performance improvements of up to 19.5%. Our code is available at https://github.com/kjh9503/caqr.

Figures (8)

• Figure 1: The existing methods may include wrong answers such as, Sprint, Marathon, Triathlon, and Figure because the candidates held by the variable node (V) in the inference process are only influenced by the 1980 Olympic and $FeaturedAt^{-1}$. However, our approach uses structural and relation-induced contexts to find a more accurate embedding of V, which helps us to predict answers that are close to the ground truth.
• Figure 2: Five types of query graph
• Figure 3: The figure of the structural context and relation-induced context and its application example on ip query. Each node in the query graph can be assigned a position number and a role number, which is represented as a tuple in the Structural Context box (green box). The first number of the tuple in each node of the query graph represents the position number, and the second number indicates the role number. Type embedding is derived from the query-type table containing the position and role information of the corresponding query graph. The Relation-induced Context box illustrates constructing relation-induced embedding of node $V$ and $A$ from KG. The Integration box describes integrating query embedding, position embedding, role embedding, type embedding, and relation-induced embedding into updated query embedding. Best viewed in color.
• Figure 4: Ablation study on the existence of Position, Role, and Type embedding. FB237 denotes the FB15k-237 dataset. +P, +Rol, +T and +S indicates the model with position embedding, role embedding, type embedding, and all the structure embeddings, respectively.
• Figure 5: Effect of Hyper-parameters on Q2B+{CaQR(S), CaQR(R)}.