Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploring Bias and Prediction Metrics to Characterise the Fairness of Machine Learning for Equity-Centered Public Health Decision-Making: A Narrative Review

Shaina Raza, Arash Shaban-Nejad, Elham Dolatabadi, Hiroshi Mamiya

TL;DR

A review of studies describing types of bias and metrics in the domain of ML and public and population health published in English between 2008 and 2023, inclusive to help formalize the evaluation framework for ML on public health from an equity perspective.

Abstract

Background: The rapid advancement of Machine Learning (ML) represents novel opportunities to enhance public health research, surveillance, and decision-making. However, there is a lack of comprehensive understanding of algorithmic bias, systematic errors in predicted population health outcomes, resulting from the public health application of ML. The objective of this narrative review is to explore the types of bias generated by ML and quantitative metrics to assess these biases. Methods : We performed search on PubMed, MEDLINE, IEEE (Institute of Electrical and Electronics Engineers), ACM (Association for Computing Machinery) Digital Library, Science Direct, and Springer Nature. We used keywords to identify studies describing types of bias and metrics to measure these in the domain of ML and public and population health published in English between 2008 and 2023, inclusive. Results: A total of 72 articles met the inclusion criteria. Our review identified the commonly described types of bias and quantitative metrics to assess these biases from an equity perspective. Conclusion : The review will help formalize the evaluation framework for ML on public health from an equity perspective.

Exploring Bias and Prediction Metrics to Characterise the Fairness of Machine Learning for Equity-Centered Public Health Decision-Making: A Narrative Review

TL;DR

A review of studies describing types of bias and metrics in the domain of ML and public and population health published in English between 2008 and 2023, inclusive to help formalize the evaluation framework for ML on public health from an equity perspective.

Abstract

Background: The rapid advancement of Machine Learning (ML) represents novel opportunities to enhance public health research, surveillance, and decision-making. However, there is a lack of comprehensive understanding of algorithmic bias, systematic errors in predicted population health outcomes, resulting from the public health application of ML. The objective of this narrative review is to explore the types of bias generated by ML and quantitative metrics to assess these biases. Methods : We performed search on PubMed, MEDLINE, IEEE (Institute of Electrical and Electronics Engineers), ACM (Association for Computing Machinery) Digital Library, Science Direct, and Springer Nature. We used keywords to identify studies describing types of bias and metrics to measure these in the domain of ML and public and population health published in English between 2008 and 2023, inclusive. Results: A total of 72 articles met the inclusion criteria. Our review identified the commonly described types of bias and quantitative metrics to assess these biases from an equity perspective. Conclusion : The review will help formalize the evaluation framework for ML on public health from an equity perspective.
Paper Structure (30 sections, 3 figures, 3 tables)

This paper contains 30 sections, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Public Health Utilities of Machine Learning
  3. Ethical Implications of Utilizing Machine Learning
  4. Fairness and Equity in Machine Learning Applications
  5. Research Aim and Objective
  6. Methods
  7. Databases Used
  8. Search Terms
  9. Article Selection Strategy
  10. Results
  11. Biases impacting fairness of ML algorithms
  12. Data Quality and Diversity
  13. Algorithmic Processing
  14. Cultural and Social Context
  15. Human-AI Interaction
  16. ...and 15 more sections

Figures (3)

  • Figure 1: Schematic Representation of Outcome Distribution by Race and Gender. This diagram illustrates the distribution of positive (blue and bordered) and negative (red) outcomes from an ML model for two groups, R1 and R2, representing white and black males, respectively. Notably, the classifier exhibits higher diagnostic accuracy for the condition in patients from the blue group R1 (White males) compared to the red group R2 (Black males). The dashed line represents the class boundary separating the outcomes for the two groups, visually emphasizing the disparity in accuracy between the groups.
  • Figure 2: Flow diagram showing the process of study selection.
  • Figure 3: Various sources and categories of biases within the public health context mapped to concepts (rounded rectangles)