Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Counting Still Counts: Understanding Neural Complex Query Answering Through Query Relaxation

Yannick Brunink, Daniel Daza, Yunjie He, Michael Cochez

TL;DR

This work questions whether neural complex query answering (CQA) over knowledge graphs truly extends beyond symbolic reasoning by contrasting state-of-the-art neural CQA models with a training-free query relaxation baseline that counts matching paths. Across diverse datasets and query types, neural methods do not consistently surpass RELAX, and the top answers differ substantially between approaches, indicating complementary reasoning signals. An oracle-based combination reveals substantial gains, especially for longer path queries, underscoring the value of integrating relaxation-like structural reasoning with neural methods. The findings call for stronger non-neural baselines and pave the way for hybrid CQA models that merge symbolic relaxation with learned representations.

Abstract

Neural methods for Complex Query Answering (CQA) over knowledge graphs (KGs) are widely believed to learn patterns that generalize beyond explicit graph structure, allowing them to infer answers that are unreachable through symbolic query processing. In this work, we critically examine this assumption through a systematic analysis comparing neural CQA models with an alternative, training-free query relaxation strategy that retrieves possible answers by relaxing query constraints and counting resulting paths. Across multiple datasets and query structures, we find several cases where neural and relaxation-based approaches perform similarly, with no neural model consistently outperforming the latter. Moreover, a similarity analysis reveals that their retrieved answers exhibit little overlap, and that combining their outputs consistently improves performance. These results call for a re-evaluation of progress in neural query answering: despite their complexity, current models fail to subsume the reasoning patterns captured by query relaxation. Our findings highlight the importance of stronger non-neural baselines and suggest that future neural approaches could benefit from incorporating principles of query relaxation.

Counting Still Counts: Understanding Neural Complex Query Answering Through Query Relaxation

TL;DR

This work questions whether neural complex query answering (CQA) over knowledge graphs truly extends beyond symbolic reasoning by contrasting state-of-the-art neural CQA models with a training-free query relaxation baseline that counts matching paths. Across diverse datasets and query types, neural methods do not consistently surpass RELAX, and the top answers differ substantially between approaches, indicating complementary reasoning signals. An oracle-based combination reveals substantial gains, especially for longer path queries, underscoring the value of integrating relaxation-like structural reasoning with neural methods. The findings call for stronger non-neural baselines and pave the way for hybrid CQA models that merge symbolic relaxation with learned representations.

Abstract

Neural methods for Complex Query Answering (CQA) over knowledge graphs (KGs) are widely believed to learn patterns that generalize beyond explicit graph structure, allowing them to infer answers that are unreachable through symbolic query processing. In this work, we critically examine this assumption through a systematic analysis comparing neural CQA models with an alternative, training-free query relaxation strategy that retrieves possible answers by relaxing query constraints and counting resulting paths. Across multiple datasets and query structures, we find several cases where neural and relaxation-based approaches perform similarly, with no neural model consistently outperforming the latter. Moreover, a similarity analysis reveals that their retrieved answers exhibit little overlap, and that combining their outputs consistently improves performance. These results call for a re-evaluation of progress in neural query answering: despite their complexity, current models fail to subsume the reasoning patterns captured by query relaxation. Our findings highlight the importance of stronger non-neural baselines and suggest that future neural approaches could benefit from incorporating principles of query relaxation.

Paper Structure

This paper contains 33 sections, 11 equations, 2 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Complex Query Answering.
  4. Statistics for predictions in KGs.
  5. Query relaxation and path heuristics.
  6. Preliminaries
  7. KGs.
  8. First-order logic queries.
  9. CQA via Query Relaxation
  10. Experiments
  11. Methodology
  12. Datasets.
  13. Evaluation metrics.
  14. Query relaxation.
  15. Baselines
  16. ...and 18 more sections

Figures (2)

  • Figure 1: Query graphs for different types of conjunctive queries and their shorthand notation.
  • Figure 2: Jaccard similarity of top-$k$ retrieved answers between RELAX and three neural methods (columns), across different datasets (rows) and query structures (shown in colors in the legend). If the top-k contains the same set of answers, the similarity would reach 1.