Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Addressing cognitive bias in medical language models

Samuel Schmidgall, Carl Harris, Ime Essien, Daniel Olshvang, Tawsifur Rahman, Ji Woong Kim, Rojin Ziaei, Jason Eshraghian, Peter Abadir, Rama Chellappa

TL;DR

BiasMedQA, a benchmark for evaluating cognitive biases in LLMs applied to medical tasks, reveals varying effects for biases on these LLMs, with GPT-4 standing out for its resilience to bias, in contrast to Llama 2 70B-chat and PMC Llama 13B, which were disproportionately affected by cognitive bias.

Abstract

There is increasing interest in the application large language models (LLMs) to the medical field, in part because of their impressive performance on medical exam questions. While promising, exam questions do not reflect the complexity of real patient-doctor interactions. In reality, physicians' decisions are shaped by many complex factors, such as patient compliance, personal experience, ethical beliefs, and cognitive bias. Taking a step toward understanding this, our hypothesis posits that when LLMs are confronted with clinical questions containing cognitive biases, they will yield significantly less accurate responses compared to the same questions presented without such biases. In this study, we developed BiasMedQA, a benchmark for evaluating cognitive biases in LLMs applied to medical tasks. Using BiasMedQA we evaluated six LLMs, namely GPT-4, Mixtral-8x70B, GPT-3.5, PaLM-2, Llama 2 70B-chat, and the medically specialized PMC Llama 13B. We tested these models on 1,273 questions from the US Medical Licensing Exam (USMLE) Steps 1, 2, and 3, modified to replicate common clinically-relevant cognitive biases. Our analysis revealed varying effects for biases on these LLMs, with GPT-4 standing out for its resilience to bias, in contrast to Llama 2 70B-chat and PMC Llama 13B, which were disproportionately affected by cognitive bias. Our findings highlight the critical need for bias mitigation in the development of medical LLMs, pointing towards safer and more reliable applications in healthcare.

Addressing cognitive bias in medical language models

TL;DR

BiasMedQA, a benchmark for evaluating cognitive biases in LLMs applied to medical tasks, reveals varying effects for biases on these LLMs, with GPT-4 standing out for its resilience to bias, in contrast to Llama 2 70B-chat and PMC Llama 13B, which were disproportionately affected by cognitive bias.

Abstract

There is increasing interest in the application large language models (LLMs) to the medical field, in part because of their impressive performance on medical exam questions. While promising, exam questions do not reflect the complexity of real patient-doctor interactions. In reality, physicians' decisions are shaped by many complex factors, such as patient compliance, personal experience, ethical beliefs, and cognitive bias. Taking a step toward understanding this, our hypothesis posits that when LLMs are confronted with clinical questions containing cognitive biases, they will yield significantly less accurate responses compared to the same questions presented without such biases. In this study, we developed BiasMedQA, a benchmark for evaluating cognitive biases in LLMs applied to medical tasks. Using BiasMedQA we evaluated six LLMs, namely GPT-4, Mixtral-8x70B, GPT-3.5, PaLM-2, Llama 2 70B-chat, and the medically specialized PMC Llama 13B. We tested these models on 1,273 questions from the US Medical Licensing Exam (USMLE) Steps 1, 2, and 3, modified to replicate common clinically-relevant cognitive biases. Our analysis revealed varying effects for biases on these LLMs, with GPT-4 standing out for its resilience to bias, in contrast to Llama 2 70B-chat and PMC Llama 13B, which were disproportionately affected by cognitive bias. Our findings highlight the critical need for bias mitigation in the development of medical LLMs, pointing towards safer and more reliable applications in healthcare.
Paper Structure (8 sections, 3 figures, 5 tables)

This paper contains 8 sections, 3 figures, 5 tables.

Table of Contents

  1. Tabular results
  2. Prompting strategies
  3. Prompts without bias mitigation
  4. Education mitigation
  5. One-shot mitigation strategy
  6. Few-shot mitigation strategy
  7. Non-response
  8. Automated evaluation

Figures (3)

  • Figure 1: Demonstration of language model interaction scenario given questions from the US Medical Licensing Exam. (Left) Graphical depiction of language model interaction. (Middle) Textual depiction of unbiased prompt for LLM. (Right) Prompt with example of cognitive bias prompt.
  • Figure 2: Model comparison following cognitive bias addition. Accuracy is indicated by the distance between each dot and the origin (e.g., a radius of 0.8 corresponds to 80% accuracy). The names of each cognitive bias surround the circle. Table \ref{['tab:no_mitigation']} shows the results in tabular format.
  • Figure 3: Mitigation strategy performance. Model names are shown above each radar plot. Tables \ref{['tab:no_mitigation']}-\ref{['tab:few_shot']} show the results in tabular format.