Conditional Logical Message Passing Transformer for Complex Query Answering

TL;DR

This work tackles Complex Query Answering on incomplete knowledge graphs by introducing Conditional Logical Message Passing Transformer (CLMPT). CLMPT performs message passing conditionally on node types (constants vs. variables) and uses a transformer encoder to adaptively weight and integrate logical messages for variable nodes, enabling dynamic modeling of implicit logical dependencies. By combining one-hop logical messages derived from pre-trained neural link predictors with a transformer-based update, CLMPT achieves state-of-the-art results on EPFO queries across FB15k, FB15k-237, and NELL995, while reducing unnecessary computation. The approach demonstrates strong performance advantages, scalable computation, and offers a pathway to further improvements by incorporating symbolic information for negative queries future work.

Abstract

Complex Query Answering (CQA) over Knowledge Graphs (KGs) is a challenging task. Given that KGs are usually incomplete, neural models are proposed to solve CQA by performing multi-hop logical reasoning. However, most of them cannot perform well on both one-hop and multi-hop queries simultaneously. Recent work proposes a logical message passing mechanism based on the pre-trained neural link predictors. While effective on both one-hop and multi-hop queries, it ignores the difference between the constant and variable nodes in a query graph. In addition, during the node embedding update stage, this mechanism cannot dynamically measure the importance of different messages, and whether it can capture the implicit logical dependencies related to a node and received messages remains unclear. In this paper, we propose Conditional Logical Message Passing Transformer (CLMPT), which considers the difference between constants and variables in the case of using pre-trained neural link predictors and performs message passing conditionally on the node type. We empirically verified that this approach can reduce computational costs without affecting performance. Furthermore, CLMPT uses the transformer to aggregate received messages and update the corresponding node embedding. Through the self-attention mechanism, CLMPT can assign adaptive weights to elements in an input set consisting of received messages and the corresponding node and explicitly model logical dependencies between various elements. Experimental results show that CLMPT is a new state-of-the-art neural CQA model. https://github.com/qianlima-lab/CLMPT.

Figures (3)

• Figure 1: EFO-1 query and its query graph for the question “Who starred the films that were directed by Mizoguchi but never won a Venice Film Festival Award?”.
• Figure 2: An illustration of the two-stage procedures of CLMPT at the $l$-th layer. (a) Passing the logical message to the variable node in the neighborhood. (b) Updating the existential variable node embedding with the received messages and a transformer encoder. Similarly, such a node embedding update scheme is applied to other variable nodes, such as the free variable node $y$.
• Figure 3: All the query types considered in our experiments, where $p$, $i$, $u$, and $n$ represent projection, intersection, union, and negation, respectively. The naming of each query type reflects how they were generated in the BetaE paper ren2020beta.