Pregnant Questions: The Importance of Pragmatic Awareness in Maternal Health Question Answering

TL;DR

This work addresses the problem of latent false assumptions in maternal-health questions by introducing a dataset of 2,727 pragmatic inferences drawn from 500 questions across three sources (rosie, Reddit, Natural Questions). It grounds presupposition and implicature in linguistic theory and shows that health experts naturally address many of these inferences in their answers. By augmenting QA pipelines with pragmatic inferences, the study achieves competitive automatic metrics and often improved human judgments, especially for highly plausible false inferences, demonstrating a path toward safer, more complete maternal-health QA. The findings imply that future real-world QA systems in high-stakes domains should proactively interrogate the pragmatics of user questions to mitigate harmful beliefs and improve information quality.

Abstract

Questions posed by information-seeking users often contain implicit false or potentially harmful assumptions. In a high-risk domain such as maternal and infant health, a question-answering system must recognize these pragmatic constraints and go beyond simply answering user questions, examining them in context to respond helpfully. To achieve this, we study assumptions and implications, or pragmatic inferences, made when mothers ask questions about pregnancy and infant care by collecting a dataset of 2,727 inferences from 500 questions across three diverse sources. We study how health experts naturally address these inferences when writing answers, and illustrate that informing existing QA pipelines with pragmatic inferences produces responses that are more complete, mitigating the propagation of harmful beliefs.

Figures (6)

• Figure 1: We ask public health experts to identify assumptions and implications from questions and find that incorporating them in a qa pipeline produces a more complete answer.
• Figure 2: Tree-map shneiderman1992tree visualizing the distribution of expert-annotated pragmatic inferences in inference-sample with their veracity, inference type, and whether or not they were addressed or not in the expert-written answer to the question from which they came from. When users make false or subjective inferences, they are more likely to do so as an implicature. Moreover, when an inference is false, it is more likely to be naturally addressed in an answer by public health experts.
• Figure 3: Our baseline and pragmatic inference-augmented qa pipelines. We experiment with retrieval, reranking, and reading stages and a variety of instruction-tuned and prompt-based models.
• Figure 4: Ratings of expert validators of the plausibility of inferences written by health experts in our dataset. The majority of inferences are plausible.
• Figure 5: Human evaluation instructions provided to two expert annotators.