Question Generation in Knowledge-Driven Dialog: Explainability and Evaluation

TL;DR

This work tackles question generation in knowledge-grounded dialogs by conditioning on a predicted knowledge-base triple $f_q$ before producing the question $q$. The authors introduce the Extended QG model, which outputs the triple–question pair $f_q,q$ and compare it to a standard Question-Only QG model, demonstrating that the triple-conditioned approach offers explainability and enables reference-free evaluation while achieving performance on par with the baselines. Evaluations on the KGConv-derived data show high factuality and relevance of generated triples, strong triple–question alignment as measured by GLEU and G-BLEU, and useful insights into pronoun use and ambiguity. Ablation studies reveal that both the knowledge graph and the dialog context substantially improve coherence and reduce off-topic or hallucinated outputs. The results suggest a practical path toward interpretable, knowledge-grounded dialog systems and point to future work on broader KB-grounded tasks such as CSQA and improved pronoun resolution.

Abstract

We explore question generation in the context of knowledge-grounded dialogs focusing on explainability and evaluation. Inspired by previous work on planning-based summarisation, we present a model which instead of directly generating a question, sequentially predicts first a fact then a question. We evaluate our approach on 37k test dialogs adapted from the KGConv dataset and we show that, although more demanding in terms of inference, our approach performs on par with a standard model which solely generates a question while allowing for a detailed referenceless evaluation of the model behaviour in terms of relevance, factuality and pronominalisation.

Figures (2)

• Figure 1: Example of Gender Ambiguous Pronoun: The pronoun denotes a male entity (William Herschel) which is different from the last mentioned male entity (Nevil Maskelyne).
• Figure 2: Examples of triples generated when ablating RDF graph $K$.