Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Analyzing Diversity in Healthcare LLM Research: A Scientometric Perspective

David Restrepo, Chenwei Wu, Constanza Vásquez-Venegas, João Matos, Jack Gallifant, Leo Anthony Celi, Danielle S. Bitterman, Luis Filipe Nakayama

TL;DR

This scientometric study analyzes diversity in healthcare LLM research from 2021 to 2024 using PubMed and Dimensions metadata to quantify gender, geographic, and income-group representation. It introduces a Gini-based journal diversity index and applies bootstrap methods to estimate uncertainty, revealing pronounced gender gaps and heavy concentration of output and funding in high-income regions, particularly North America and Europe. The findings highlight substantial underrepresentation of LMICs and women, with variability across journals in inclusivity. The paper offers concrete recommendations for funding, collaboration, and publishing policy to foster more equitable and globally applicable AI solutions in healthcare.

Abstract

The deployment of large language models (LLMs) in healthcare has demonstrated substantial potential for enhancing clinical decision-making, administrative efficiency, and patient outcomes. However, the underrepresentation of diverse groups in the development and application of these models can perpetuate biases, leading to inequitable healthcare delivery. This paper presents a comprehensive scientometric analysis of LLM research for healthcare, including data from January 1, 2021, to July 1, 2024. By analyzing metadata from PubMed and Dimensions, including author affiliations, countries, and funding sources, we assess the diversity of contributors to LLM research. Our findings highlight significant gender and geographic disparities, with a predominance of male authors and contributions primarily from high-income countries (HICs). We introduce a novel journal diversity index based on Gini diversity to measure the inclusiveness of scientific publications. Our results underscore the necessity for greater representation in order to ensure the equitable application of LLMs in healthcare. We propose actionable strategies to enhance diversity and inclusivity in artificial intelligence research, with the ultimate goal of fostering a more inclusive and equitable future in healthcare innovation.

Analyzing Diversity in Healthcare LLM Research: A Scientometric Perspective

TL;DR

This scientometric study analyzes diversity in healthcare LLM research from 2021 to 2024 using PubMed and Dimensions metadata to quantify gender, geographic, and income-group representation. It introduces a Gini-based journal diversity index and applies bootstrap methods to estimate uncertainty, revealing pronounced gender gaps and heavy concentration of output and funding in high-income regions, particularly North America and Europe. The findings highlight substantial underrepresentation of LMICs and women, with variability across journals in inclusivity. The paper offers concrete recommendations for funding, collaboration, and publishing policy to foster more equitable and globally applicable AI solutions in healthcare.

Abstract

The deployment of large language models (LLMs) in healthcare has demonstrated substantial potential for enhancing clinical decision-making, administrative efficiency, and patient outcomes. However, the underrepresentation of diverse groups in the development and application of these models can perpetuate biases, leading to inequitable healthcare delivery. This paper presents a comprehensive scientometric analysis of LLM research for healthcare, including data from January 1, 2021, to July 1, 2024. By analyzing metadata from PubMed and Dimensions, including author affiliations, countries, and funding sources, we assess the diversity of contributors to LLM research. Our findings highlight significant gender and geographic disparities, with a predominance of male authors and contributions primarily from high-income countries (HICs). We introduce a novel journal diversity index based on Gini diversity to measure the inclusiveness of scientific publications. Our results underscore the necessity for greater representation in order to ensure the equitable application of LLMs in healthcare. We propose actionable strategies to enhance diversity and inclusivity in artificial intelligence research, with the ultimate goal of fostering a more inclusive and equitable future in healthcare innovation.
Paper Structure (22 sections, 8 equations, 1 figure, 6 tables)

This paper contains 22 sections, 8 equations, 1 figure, 6 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Cohort Selection
  4. Metadata Extraction and Preprocessing
  5. Data Analysis
  6. Gender Distributions
  7. Continent and Income Group Distributions per author and funding source
  8. Distribution of Publications per Country:
  9. Journal Diversity and Index Testing:
  10. Results
  11. Obtained Cohort
  12. Gender Distributions
  13. Distribution of Papers per Continent for LMICs and HICs
  14. Funding Source Analysis
  15. Journal Diversity Index
  16. ...and 7 more sections

Figures (1)

  • Figure 1: World Map Distribution of Research Contributions