Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Transformers for Complex Query Answering over Knowledge Hypergraphs

Hong Ting Tsang, Zihao Wang, Yangqiu Song

TL;DR

This work tackles complex query answering over knowledge hypergraphs (KHGs) with n-ary relations, where traditional binary-graph methods fall short. It introduces LKHGT, a two-stage Transformer with a Projection Encoder and a Logical Encoder, both equipped with Type Aware Bias to handle heterogeneous token interactions, and it demonstrates that Transformer-based logical reasoning can replace fuzzy logic in CQA. The authors also construct two HCQA datasets, JF17k-HCQA and M-FB15k-HCQA, covering 14 query types under existential first-order logic (EFO-1), and show state-of-the-art results on these tasks, including generalization to out-of-distribution queries. Overall, LKHGT bridges Complex Query Answering and Knowledge Hypergraphs by enabling robust, multi-hop reasoning over ordered hyperedges, with implications for scalable reasoning on real-world multi-ary knowledge graphs.

Abstract

Complex Query Answering (CQA) has been extensively studied in recent years. In order to model data that is closer to real-world distribution, knowledge graphs with different modalities have been introduced. Triple KGs, as the classic KGs composed of entities and relations of arity 2, have limited representation of real-world facts. Real-world data is more sophisticated. While hyper-relational graphs have been introduced, there are limitations in representing relationships of varying arity that contain entities with equal contributions. To address this gap, we sampled new CQA datasets: JF17k-HCQA and M-FB15k-HCQA. Each dataset contains various query types that include logical operations such as projection, negation, conjunction, and disjunction. In order to answer knowledge hypergraph (KHG) existential first-order queries, we propose a two-stage transformer model, the Logical Knowledge Hypergraph Transformer (LKHGT), which consists of a Projection Encoder for atomic projection and a Logical Encoder for complex logical operations. Both encoders are equipped with Type Aware Bias (TAB) for capturing token interactions. Experimental results on CQA datasets show that LKHGT is a state-of-the-art CQA method over KHG and is able to generalize to out-of-distribution query types.

Transformers for Complex Query Answering over Knowledge Hypergraphs

TL;DR

This work tackles complex query answering over knowledge hypergraphs (KHGs) with n-ary relations, where traditional binary-graph methods fall short. It introduces LKHGT, a two-stage Transformer with a Projection Encoder and a Logical Encoder, both equipped with Type Aware Bias to handle heterogeneous token interactions, and it demonstrates that Transformer-based logical reasoning can replace fuzzy logic in CQA. The authors also construct two HCQA datasets, JF17k-HCQA and M-FB15k-HCQA, covering 14 query types under existential first-order logic (EFO-1), and show state-of-the-art results on these tasks, including generalization to out-of-distribution queries. Overall, LKHGT bridges Complex Query Answering and Knowledge Hypergraphs by enabling robust, multi-hop reasoning over ordered hyperedges, with implications for scalable reasoning on real-world multi-ary knowledge graphs.

Abstract

Complex Query Answering (CQA) has been extensively studied in recent years. In order to model data that is closer to real-world distribution, knowledge graphs with different modalities have been introduced. Triple KGs, as the classic KGs composed of entities and relations of arity 2, have limited representation of real-world facts. Real-world data is more sophisticated. While hyper-relational graphs have been introduced, there are limitations in representing relationships of varying arity that contain entities with equal contributions. To address this gap, we sampled new CQA datasets: JF17k-HCQA and M-FB15k-HCQA. Each dataset contains various query types that include logical operations such as projection, negation, conjunction, and disjunction. In order to answer knowledge hypergraph (KHG) existential first-order queries, we propose a two-stage transformer model, the Logical Knowledge Hypergraph Transformer (LKHGT), which consists of a Projection Encoder for atomic projection and a Logical Encoder for complex logical operations. Both encoders are equipped with Type Aware Bias (TAB) for capturing token interactions. Experimental results on CQA datasets show that LKHGT is a state-of-the-art CQA method over KHG and is able to generalize to out-of-distribution query types.

Paper Structure

This paper contains 27 sections, 11 equations, 5 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Knowledge Hypergraphs Embedding
  4. N-ary graph Message Passing
  5. Complex Query Answering
  6. Transformer in CQA
  7. N-ary Graph Reasoning
  8. PRELIMINARIES
  9. Knowledge Hypergraphs
  10. EFO-1 Query in Hypergraphs
  11. Logical Knowledge Hyper Graph Transformer
  12. Tokens Input and Output
  13. Guided attention for Projection Encoder and Logical Encoder
  14. Transformer as the replacement for Fuzzy Logic
  15. Training LKHGT
  16. ...and 12 more sections

Figures (5)

  • Figure 1: Operator Tree
  • Figure 2: Hyperedge Query Graph
  • Figure 3: Example of LKHGT processing query tree
  • Figure 4: Projection Enconder
  • Figure 5: Logical Encoder

Theorems & Definitions (2)

  • Definition 1
  • Definition 2