A Women's Health Benchmark for Large Language Models

TL;DR

The paper introduces the Women’s Health Benchmark (WHB), the first targeted evaluation suite for assessing large language models on women’s health topics. It crowdsources 96 realistic, expert-annotated prompts across five specialties and three query types, classifying responses into eight error types with justification and sources. Evaluating 13 state-of-the-art LLMs, the study finds an average failure rate around 60%, with performance varying by specialty, query type, and error category; especially, missed urgency is a persistent weakness, though newer models like GPT-5 show meaningful improvements in preventing inappropriate recommendations. The work highlights the urgent need for sex-aware data and evaluation in health AI, advocates for larger, multi-turn benchmarks, and provides a publicly available dataset to catalyze ongoing progress toward safer, more reliable women’s health guidance from AI systems.

Abstract

As large language models (LLMs) become primary sources of health information for millions, their accuracy in women's health remains critically unexamined. We introduce the Women's Health Benchmark (WHB), the first benchmark evaluating LLM performance specifically in women's health. Our benchmark comprises 96 rigorously validated model stumps covering five medical specialties (obstetrics and gynecology, emergency medicine, primary care, oncology, and neurology), three query types (patient query, clinician query, and evidence/policy query), and eight error types (dosage/medication errors, missing critical information, outdated guidelines/treatment recommendations, incorrect treatment advice, incorrect factual information, missing/incorrect differential diagnosis, missed urgency, and inappropriate recommendations). We evaluated 13 state-of-the-art LLMs and revealed alarming gaps: current models show approximately 60\% failure rates on the women's health benchmark, with performance varying dramatically across specialties and error types. Notably, models universally struggle with "missed urgency" indicators, while newer models like GPT-5 show significant improvements in avoiding inappropriate recommendations. Our findings underscore that AI chatbots are not yet fully able of providing reliable advice in women's health.

Figures (6)

• Figure 1: Overview of WHB methodology. Top: Dataset collection workflow showing how the expert cohort generated women's health-related clinical prompts, which were randomly assigned to one of the 13 LLMs to identify incorrect responses that became model stumps. Bottom: Model performance evaluation workflow showing how the 96 model stumps were used to benchmark all 13 LLMs.
• Figure 2: Distribution of model stumps in WHB. (A) Distribution by query type. (B) Distribution by medical specialty. (C) Distribution by error type.
• Figure 3: Model performance on WHB. The approval rate was calculated as the percentage of correct cases (correct / total number of cases), shown in green columns. The failure rate was calculated as the percentage of incorrect cases (incorrect/total number of cases), shown in red columns.
• Figure 4: Model performance across medical specialties. (A) Overall failure rates (in percentage) by medical specialty across all models with 95% confidence intervals. (B) Heatmap showing failure rates by model and medical specialty. Lower values (green) indicate better performance, while higher values (red) indicate worse performance.
• Figure 5: Model performance across error types. (A) Overall failure rates (in percentage) by error type across all models with 95% confidence intervals. (B) Heatmap showing failure rates by model and error type.