Answer-based Adversarial Training for Generating Clarification Questions
Sudha Rao, Hal Daumé
TL;DR
This work tackles generating clarification questions that elicit new information to make textual contexts more complete. It introduces an adversarial framework where a sequence-to-sequence question generator is guided by a latent, hypothetical answer and a Utility-based discriminator, with Mixer-based training to optimize a usefulness reward. The approach, GAN-Utility, and its ablations are evaluated on Amazon product descriptions and Stack Exchange posts using automatic metrics and human judgments, showing improved usefulness and specificity over baselines and retrieval methods. The results demonstrate the potential of answer-based adversarial training to produce more targeted and informative clarification questions, with implications for enhancing knowledge base completion, product Q&A, and discussion forums. The work also discusses limitations of automatic metrics for text and points to future directions including multi-modal inputs and deployment in real platforms.
Abstract
We present an approach for generating clarification questions with the goal of eliciting new information that would make the given textual context more complete. We propose that modeling hypothetical answers (to clarification questions) as latent variables can guide our approach into generating more useful clarification questions. We develop a Generative Adversarial Network (GAN) where the generator is a sequence-to-sequence model and the discriminator is a utility function that models the value of updating the context with the answer to the clarification question. We evaluate on two datasets, using both automatic metrics and human judgments of usefulness, specificity and relevance, showing that our approach outperforms both a retrieval-based model and ablations that exclude the utility model and the adversarial training.