What if you said that differently?: How Explanation Formats Affect Human Feedback Efficacy and User Perception

TL;DR

This work investigates how the format of intermediate rationales in a decomposed QA system influences the ease of repairing model outputs via human feedback and the perceived interpretability and trustworthiness of model answers. It introduces five rationale formats (Markup-and-Mask, Annotated Report, Procedural, Subquestions, and Decision Tree) and evaluates them through two user studies on Quoref and PubMedQA. Key findings show that formats exposing reasoning and including attributions (notably Annotated Report) improve user understanding and trust, while some formats facilitate easier repair in different domains; attribution and depth of reasoning are highly valued. The results offer practical guidance for designing human-in-the-loop QA systems and collecting feedback from end users to enhance model reliability and user confidence.

Abstract

Eliciting feedback from end users of NLP models can be beneficial for improving models. However, how should we present model responses to users so they are most amenable to be corrected from user feedback? Further, what properties do users value to understand and trust responses? We answer these questions by analyzing the effect of rationales (or explanations) generated by QA models to support their answers. We specifically consider decomposed QA models that first extract an intermediate rationale based on a context and a question and then use solely this rationale to answer the question. A rationale outlines the approach followed by the model to answer the question. Our work considers various formats of these rationales that vary according to well-defined properties of interest. We sample rationales from language models using few-shot prompting for two datasets, and then perform two user studies. First, we present users with incorrect answers and corresponding rationales in various formats and ask them to provide natural language feedback to revise the rationale. We then measure the effectiveness of this feedback in patching these rationales through in-context learning. The second study evaluates how well different rationale formats enable users to understand and trust model answers, when they are correct. We find that rationale formats significantly affect how easy it is (1) for users to give feedback for rationales, and (2) for models to subsequently execute this feedback. In addition, formats with attributions to the context and in-depth reasoning significantly enhance user-reported understanding and trust of model outputs.

Figures (7)

• Figure 1: The framework for incorporating human feedback into decomposed QA models. A model $X2R$ generates a rationale $R$ to answer a question based on a passage. A human teacher then provides natural language feedback for $R$, which is used to generate a revised rationale $R'$ from $F2R'$. Finally, this revised rationale is used to generate the final answer $Y$. We study various formats of the intermediate rationale $R$.
• Figure 2: Examples of the different rationale formats considered for representing intermediate rationales.
• Figure 3: Likert distribution of the annotator judgements of interpretability & trustworthiness for different rationale formats corresponding to correct answers (§\ref{['sec:study2']}).
• Figure 4: Scalar judgements of characteristics that annotators value in intermediate rationales (scale of 1-5).
• Figure 5: Screenshots of the interface (1).