Question Rewriting for Conversational Question Answering

TL;DR

The paper tackles conversational question answering by decomposing the task into Question Rewriting (QR) and standard QA. It introduces a unidirectional Transformer QR model that rewrites ambiguous follow-up questions into explicit ones, enabling existing non-conversational QA systems to operate in a conversational setting. The proposed approach yields state-of-the-art retrieval on TREC CAsT and improves extractive QA on QuAC-derived CANARD, with an error analysis showing QA as the primary bottleneck. The framework offers traceability and reuse across QA architectures and points to future work on joint retrieval–answer models and multi-modal context integration.

Abstract

Conversational question answering (QA) requires the ability to correctly interpret a question in the context of previous conversation turns. We address the conversational QA task by decomposing it into question rewriting and question answering subtasks. The question rewriting (QR) subtask is specifically designed to reformulate ambiguous questions, which depend on the conversational context, into unambiguous questions that can be correctly interpreted outside of the conversational context. We introduce a conversational QA architecture that sets the new state of the art on the TREC CAsT 2019 passage retrieval dataset. Moreover, we show that the same QR model improves QA performance on the QuAC dataset with respect to answer span extraction, which is the next step in QA after passage retrieval. Our evaluation results indicate that the QR model we proposed achieves near human-level performance on both datasets and the gap in performance on the end-to-end conversational QA task is attributed mostly to the errors in QA.

Figures (6)

• Figure 1: Our approach for end-to-end conversational QA relies on the question rewriting component to handle conversation context and produce an explicit question that can be fed to standard, non-conversational QA components.
• Figure 2: The question rewriting component uses the Transformer Decoder architecture, to recursively generate the tokens of an "explicit" question. At inference time, the generated output is appended to the input sequence for the next timestep in the sequence.
• Figure 3: Retrieval QA component includes two sequential phases: candidate selection (BM25) followed by passage re-ranking (Transformer Encoder).
• Figure 4: Extractive QA component predicts a span of text in the paragraph P', given an input sequence with the question Q' and passage P'.
• Figure 5: Effect from fine-tuning the MultiQA model on a portion of the target CANARD-H dataset due to the domain shift between the datasets.