Effective Instruction Parsing Plugin for Complex Logical Query Answering on Knowledge Graphs

TL;DR

An effective Query Instruction Parsing Plugin (QIPP) that leverages the context awareness of Pre-trained Language Models (PLMs) to capture latent query patterns from code-like query instructions and a query-guided instruction decoder to adapt query patterns to KGQE models is proposed.

Abstract

Knowledge Graph Query Embedding (KGQE) aims to embed First-Order Logic (FOL) queries in a low-dimensional KG space for complex reasoning over incomplete KGs. To enhance the generalization of KGQE models, recent studies integrate various external information (such as entity types and relation context) to better capture the logical semantics of FOL queries. The whole process is commonly referred to as Query Pattern Learning (QPL). However, current QPL methods typically suffer from the pattern-entity alignment bias problem, leading to the learned defective query patterns limiting KGQE models' performance. To address this problem, we propose an effective Query Instruction Parsing Plugin (QIPP) that leverages the context awareness of Pre-trained Language Models (PLMs) to capture latent query patterns from code-like query instructions. Unlike the external information introduced by previous QPL methods, we first propose code-like instructions to express FOL queries in an alternative format. This format utilizes textual variables and nested tuples to convey the logical semantics within FOL queries, serving as raw materials for a PLM-based instruction encoder to obtain complete query patterns. Building on this, we design a query-guided instruction decoder to adapt query patterns to KGQE models. To further enhance QIPP's effectiveness across various KGQE models, we propose a query pattern injection mechanism based on compressed optimization boundaries and an adaptive normalization component, allowing KGQE models to utilize query patterns more efficiently. Extensive experiments demonstrate that our plug-and-play method improves the performance of eight basic KGQE models and outperforms two state-of-the-art QPL methods.

Figures (9)

• Figure 1: Four forms of a complex question. (a) is an NL question and (b) is the FOL query of (a). (c) and (d) are the computation graph and the code-like instruction of (b), respectively.
• Figure 2: Several query patterns illustrated by computation graphs. The projection pattern indicates that an FOL query is a single/multi-hop chain query. The intersection pattern represents that an FOL query is the intersection of multiple chain query.
• Figure 3: The overall framework of QIPP. We first transform the given question into an FOL query $q$ and a code-like instruction $X$. A specific KGQE model in the library is used to represent $q$ as the initial query embedding $\bm{f}_q$ based on the computation graph $g$. Then, we process $X$ into $\bm{X}$ through a PLM-based instruction encoder. Next, $\bm{f}_q$ is used to decode meaningful query pattern $\bm{f}_{X|q}$ from $\bm{X}$. Finally, we fuse $\bm{f}_q$ and $\bm{f}_{X|q}$ into query embedding $\bm{f}^*$ containing pattern information through a query pattern injection module and an adaptive normalization component for subsequent model training and query answering.
• Figure 4: Comparison between QIPP and different QPL plugins. Since GQE and FB15k are not used in CaQR's experiments, we only present the basic models and datasets shown in the above figure.
• Figure 5: Comparison of QIPP with different variants. Results of FB15k are provided in Figure \ref{['FigPatternInjectionFB15k']}.

Theorems & Definitions (4)

• theorem 1
• definition 1
• definition 2
• definition 3