Enhancing Complex Question Answering over Knowledge Graphs through Evidence Pattern Retrieval

TL;DR

This work addresses the noise introduced during subgraph extraction in IR-KGQA by explicitly modeling structural dependencies among evidence facts through Evidence Pattern Retrieval (EPR). EPR constructs evidence patterns from atomic adjacency patterns of resource pairs, using dense RR-AP retrieval, iterative EP construction, and a cross-encoder ranker to select the best pattern for subgraph extraction. Empirical results on ComplexWebQuestions and WebQuestionsSP show substantial gains in F1 on CWQ and competitive performance on WebQSP, demonstrating that accounting for structural dependencies improves downstream answer reasoning. The approach offers a scalable, pattern-based mechanism to improve IR-KGQA, with future work aimed at handling numerical reasoning, unseen relations, and efficiency improvements.

Abstract

Information retrieval (IR) methods for KGQA consist of two stages: subgraph extraction and answer reasoning. We argue current subgraph extraction methods underestimate the importance of structural dependencies among evidence facts. We propose Evidence Pattern Retrieval (EPR) to explicitly model the structural dependencies during subgraph extraction. We implement EPR by indexing the atomic adjacency pattern of resource pairs. Given a question, we perform dense retrieval to obtain atomic patterns formed by resource pairs. We then enumerate their combinations to construct candidate evidence patterns. These evidence patterns are scored using a neural model, and the best one is selected to extract a subgraph for downstream answer reasoning. Experimental results demonstrate that the EPR-based approach has significantly improved the F1 scores of IR-KGQA methods by over 10 points on ComplexWebQuestions and achieves competitive performance on WebQuestionsSP.

Figures (6)

• Figure 1: Facts about question "What country, containing Stahuis, does Germany border?". The evidence facts are bolded., the node of the correct answer Netherlands is underlined, and the noisy answer Austria is shaded. Austria is a noisy answer since it does not contain Stahuis, but the relations on the paths between them express similar meanings and confuse the answer reasoning model.
• Figure 2: The evidence facts (a) of question "What country, containing Stahuis, does Germany border?" and the corresponding pattern (b).
• Figure 3: Atomic patterns appeared on the evidence pattern in Figure \ref{['fig:efnep']}.
• Figure 4: Expand an under-constructed EP with an ER-AP (the left side) or an RR-AP (the right side).
• Figure 5: The performance and execution time of EPR+NSM with various numbers of APs on CWQ (a) and WebQSP (b).