CHARP: Conversation History AwaReness Probing for Knowledge-grounded Dialogue Systems

TL;DR

This work introduces CHARP, a diagnostic test set to evaluate whether knowledge-grounded dialogue systems attend to and reason over conversation history. By building eCHARP and hCHARP from FaithDial data, the authors reveal that standard FaithDial training biases models toward ignoring history, even when grounding to supplied knowledge remains faithful. The study combines automatic metrics, human judgments, and GPT-4-based evaluations to demonstrate that history-awareness is not adequately captured by existing benchmarks, and that larger open models can mitigate some issues but still struggle with history-based reasoning. The findings underscore the need for history-aware evaluation in knowledge-grounded dialogue and suggest practical paths for improving dataset design and evaluation methods.

Abstract

In this work, we dive deep into one of the popular knowledge-grounded dialogue benchmarks that focus on faithfulness, FaithDial. We show that a significant portion of the FaithDial data contains annotation artifacts, which may bias models towards completely ignoring the conversation history. We therefore introduce CHARP, a diagnostic test set, designed for an improved evaluation of hallucinations in conversational model. CHARP not only measures hallucination but also the compliance of the models to the conversation task. Our extensive analysis reveals that models primarily exhibit poor performance on CHARP due to their inability to effectively attend to and reason over the conversation history. Furthermore, the evaluation methods of FaithDial fail to capture these shortcomings, neglecting the conversational history. Our findings indicate that there is substantial room for contribution in both dataset creation and hallucination evaluation for knowledge-grounded dialogue, and that CHARP can serve as a tool for monitoring the progress in this particular research area. CHARP is publicly available at https://huggingface.co/datasets/huawei-noah/CHARP

Figures (5)

• Figure 1: CHARP consists of 2 subsets, where only the last seeker utterance differs: a self-contained easy version (eCHARP), and a hard (hCHARP) which requires reasoning over the conversation history and the provided knowledge that corresponds to the last seeker. In addition to the ground truth response annotation, we show the predictions of a model (FLAN-base) tuned on the FaithDial training data.and indicate whether the FaithDial Critic labels a response as a hallucination or not. Green boxes indicate model inputs, while pink and orange ones show predicted, and gold responses.
• Figure 2: Heatmap showing the normalized (percentage) contingency tables of evaluation categories between GPT4-turbo (rows) and human (columns) judgments. It was measured on the output of Llama-2-7B (finetuned) for both eCHARP (left) and hCHARP (right).
• Figure 3: Original example from the FaithDial validation set (left) and our edited hCHARP version (right). Green text indicates content that the model is expected to reason over, while red text marks distracting content within the provided knowledge.
• Figure 4: Heatmap showing the normalized (percentage) contingency tables of evaluation categories between GPT4-turbo (rows) and human (columns) judgments. It was measured on the output of 6 models for both eCHARP (on the left) and hCHARP (on the right).
• Figure 5: Tow examples from hCHARP (left side), along with the predictions of the six models employed in our study (right side). For each model response, we show the FaithDial judgment (hallucination indicated by , and no hallucination by ), along with the category of human judgment. In the second example, ChatGPT's response (rare but interesting) is deemed correct by human evaluators because it accurately addresses the user's comment before introducing an unrelated piece of knowledge in a manner that opens a new topic, Although it aligns with FaithDial guidelines, but the Critic judge this case as hallucination, mainly because painting new car is not mentioned in the provided knowledge.