Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLMs Do Not See Age: Assessing Demographic Bias in Automated Systematic Review Synthesis

Favour Yahdii Aghaebe, Tanefa Apekey, Elizabeth Williams, Nafise Sadat Moosavi

TL;DR

This work addresses the risk that large language models may misrepresent or omit age-related demographic information in biomedical evidence synthesis. It introduces the DemogSummary dataset and the Demographic Salience Score (DSS) to quantify age-content retention and hallucination across three models (QWEN, Longformer, GPT-4.1 Nano) and three age groups (children, adults, older adults). The study finds that adult-focused summaries exhibit the lowest demographic fidelity and highest hallucination rates, while child and older adult groups fare better, and that standard metrics fail to capture these biases. The authors advocate for fairness-aware evaluation and bias-mitigating summarisation pipelines in biomedical NLP to improve health equity and evidence validity.

Abstract

Clinical interventions often hinge on age: medications and procedures safe for adults may be harmful to children or ineffective for older adults. However, as language models are increasingly integrated into biomedical evidence synthesis workflows, it remains uncertain whether these systems preserve such crucial demographic distinctions. To address this gap, we evaluate how well state-of-the-art language models retain age-related information when generating abstractive summaries of biomedical studies. We construct DemogSummary, a novel age-stratified dataset of systematic review primary studies, covering child, adult, and older adult populations. We evaluate three prominent summarisation-capable LLMs, Qwen (open-source), Longformer (open-source) and GPT-4.1 Nano (proprietary), using both standard metrics and a newly proposed Demographic Salience Score (DSS), which quantifies age-related entity retention and hallucination. Our results reveal systematic disparities across models and age groups: demographic fidelity is lowest for adult-focused summaries, and under-represented populations are more prone to hallucinations. These findings highlight the limitations of current LLMs in faithful and bias-free summarisation and point to the need for fairness-aware evaluation frameworks and summarisation pipelines in biomedical NLP.

LLMs Do Not See Age: Assessing Demographic Bias in Automated Systematic Review Synthesis

TL;DR

This work addresses the risk that large language models may misrepresent or omit age-related demographic information in biomedical evidence synthesis. It introduces the DemogSummary dataset and the Demographic Salience Score (DSS) to quantify age-content retention and hallucination across three models (QWEN, Longformer, GPT-4.1 Nano) and three age groups (children, adults, older adults). The study finds that adult-focused summaries exhibit the lowest demographic fidelity and highest hallucination rates, while child and older adult groups fare better, and that standard metrics fail to capture these biases. The authors advocate for fairness-aware evaluation and bias-mitigating summarisation pipelines in biomedical NLP to improve health equity and evidence validity.

Abstract

Clinical interventions often hinge on age: medications and procedures safe for adults may be harmful to children or ineffective for older adults. However, as language models are increasingly integrated into biomedical evidence synthesis workflows, it remains uncertain whether these systems preserve such crucial demographic distinctions. To address this gap, we evaluate how well state-of-the-art language models retain age-related information when generating abstractive summaries of biomedical studies. We construct DemogSummary, a novel age-stratified dataset of systematic review primary studies, covering child, adult, and older adult populations. We evaluate three prominent summarisation-capable LLMs, Qwen (open-source), Longformer (open-source) and GPT-4.1 Nano (proprietary), using both standard metrics and a newly proposed Demographic Salience Score (DSS), which quantifies age-related entity retention and hallucination. Our results reveal systematic disparities across models and age groups: demographic fidelity is lowest for adult-focused summaries, and under-represented populations are more prone to hallucinations. These findings highlight the limitations of current LLMs in faithful and bias-free summarisation and point to the need for fairness-aware evaluation frameworks and summarisation pipelines in biomedical NLP.

Paper Structure

This paper contains 46 sections, 7 equations, 21 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Automatic Systematic Reviews
  4. Bias in Automated Synthesis and Summarisation
  5. The DemogSummary Dataset
  6. Dataset Construction
  7. Demographic Annotation
  8. Experimental Design
  9. Model Selection
  10. QWEN
  11. GPT
  12. Longformer
  13. Task Setup
  14. Implementation Details
  15. Evaluation Metrics
  16. ...and 31 more sections

Figures (21)

  • Figure 1: Age-stratified primary study abstracts are summarised by LLMs, and the outputs are compared to systematic review abstracts. Summaries are evaluated by age group for demographic fidelity using retention, omission, and hallucination metrics.
  • Figure 2: Subset of Gold standard Demographic Entities from Reviews
  • Figure 3: Comparison of Entity Retention and hallucinations across Models for the Adult Group - Regular Prompt.
  • Figure 4: Semantic Similarity Threshold
  • Figure 5: Alpha-Gamma Grid
  • ...and 16 more figures