Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

What Patients Really Ask: Exploring the Effect of False Assumptions in Patient Information Seeking

Raymond Xiong, Furong Jia, Lionel Wong, Monica Agrawal

TL;DR

The paper tackles the gap between exam-style medical QA benchmarks and real patient information needs by constructing a Google People Also Ask–based dataset across top prescribed medications to capture corrupted, ill-formed, and dangerous questions. It systematically labels questions and evaluates a wide range of LLMs on their ability to challenge incorrect assumptions, revealing that even leading models fail on certain queries and that misinformation can propagate along question trajectories. The work demonstrates robust statistical links between past incorrect questions and future ones, highlighting safety risks in natural user interactions. By providing a publicly available dataset and a rigorous evaluation framework, the study lays groundwork for improving LLM reliability and safety in real-world health information seeking.

Abstract

Patients are increasingly using large language models (LLMs) to seek answers to their healthcare-related questions. However, benchmarking efforts in LLMs for question answering often focus on medical exam questions, which differ significantly in style and content from the questions patients actually raise in real life. To bridge this gap, we sourced data from Google's People Also Ask feature by querying the top 200 prescribed medications in the United States, curating a dataset of medical questions people commonly ask. A considerable portion of the collected questions contains incorrect assumptions and dangerous intentions. We demonstrate that the emergence of these corrupted questions is not uniformly random and depends heavily on the degree of incorrectness in the history of questions that led to their appearance. Current LLMs that perform strongly on other benchmarks struggle to identify incorrect assumptions in everyday questions.

What Patients Really Ask: Exploring the Effect of False Assumptions in Patient Information Seeking

TL;DR

The paper tackles the gap between exam-style medical QA benchmarks and real patient information needs by constructing a Google People Also Ask–based dataset across top prescribed medications to capture corrupted, ill-formed, and dangerous questions. It systematically labels questions and evaluates a wide range of LLMs on their ability to challenge incorrect assumptions, revealing that even leading models fail on certain queries and that misinformation can propagate along question trajectories. The work demonstrates robust statistical links between past incorrect questions and future ones, highlighting safety risks in natural user interactions. By providing a publicly available dataset and a rigorous evaluation framework, the study lays groundwork for improving LLM reliability and safety in real-world health information seeking.

Abstract

Patients are increasingly using large language models (LLMs) to seek answers to their healthcare-related questions. However, benchmarking efforts in LLMs for question answering often focus on medical exam questions, which differ significantly in style and content from the questions patients actually raise in real life. To bridge this gap, we sourced data from Google's People Also Ask feature by querying the top 200 prescribed medications in the United States, curating a dataset of medical questions people commonly ask. A considerable portion of the collected questions contains incorrect assumptions and dangerous intentions. We demonstrate that the emergence of these corrupted questions is not uniformly random and depends heavily on the degree of incorrectness in the history of questions that led to their appearance. Current LLMs that perform strongly on other benchmarks struggle to identify incorrect assumptions in everyday questions.
Paper Structure (13 sections, 2 figures, 5 tables)

This paper contains 13 sections, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Data collection
  4. LLM question answering
  5. Investigation of query evolution over time
  6. Results
  7. LLM QA performance
  8. Corrupted question emergence
  9. Conclusion
  10. Data and Code Availability
  11. Additional examples of corrupted questions
  12. Prompt for LLM-as-a-judge
  13. LLM-as-a-judge alignment with human labels

Figures (2)

  • Figure 1: Google's People Also Ask feature, for search "metformin". The leftmost highlighted question assumes that doctors no longer recommend metformin, when metformin is one of the most commonly taken drugs worldwide. Clicking on a question reveals additional related questions, forming a branching structure. In each data collection trial, we followed one trajectory of clicks (shown in red) to collect questions.
  • Figure 2: Relationship between the proportion of incorrect history and the likelihood of an incorrect current question. Incorrectness is defined as either a combination of Types A and B (top) or Type A (bottom left) and Type B (bottom right) separately. Point size and color reflect the number of questions with given counts of correct and incorrect questions in history. The red curve and shaded area indicate the fitted logistic regression curve and the 95% CI region.