Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DemOpts: Fairness corrections in COVID-19 case prediction models

Naman Awasthi, Saad Abrar, Daniel Smolyak, Vanessa Frias-Martinez

TL;DR

DemOpts tackles fairness in COVID-19 county forecasts by integrating a three-step in-processing de-biasing approach into a transformer-based TFT framework. It uses seven-quantile pinball loss to model predictive errors, detects dependencies between errors and county race distributions via regression, and adjusts the loss with a parity-focused term $L_{adj}$ that activates when race–error associations are statistically significant. Compared with baseline TFT and three regression-based de-biasers (Individual, Group, Sufficiency), DemOpts improves hard and soft error parity across Asian, Black, and Hispanic counties, albeit with a modest accuracy trade-off. The work provides a practical, theoretically grounded method to mitigate bias-driven disparities in pandemic forecasting, enhancing the fairness of resource-allocation decisions informed by these models.

Abstract

COVID-19 forecasting models have been used to inform decision making around resource allocation and intervention decisions e.g., hospital beds or stay-at-home orders. State of the art deep learning models often use multimodal data such as mobility or socio-demographic data to enhance COVID-19 case prediction models. Nevertheless, related work has revealed under-reporting bias in COVID-19 cases as well as sampling bias in mobility data for certain minority racial and ethnic groups, which could in turn affect the fairness of the COVID-19 predictions along race labels. In this paper, we show that state of the art deep learning models output mean prediction errors that are significantly different across racial and ethnic groups; and which could, in turn, support unfair policy decisions. We also propose a novel de-biasing method, DemOpts, to increase the fairness of deep learning based forecasting models trained on potentially biased datasets. Our results show that DemOpts can achieve better error parity that other state of the art de-biasing approaches, thus effectively reducing the differences in the mean error distributions across more racial and ethnic groups.

DemOpts: Fairness corrections in COVID-19 case prediction models

TL;DR

DemOpts tackles fairness in COVID-19 county forecasts by integrating a three-step in-processing de-biasing approach into a transformer-based TFT framework. It uses seven-quantile pinball loss to model predictive errors, detects dependencies between errors and county race distributions via regression, and adjusts the loss with a parity-focused term that activates when race–error associations are statistically significant. Compared with baseline TFT and three regression-based de-biasers (Individual, Group, Sufficiency), DemOpts improves hard and soft error parity across Asian, Black, and Hispanic counties, albeit with a modest accuracy trade-off. The work provides a practical, theoretically grounded method to mitigate bias-driven disparities in pandemic forecasting, enhancing the fairness of resource-allocation decisions informed by these models.

Abstract

COVID-19 forecasting models have been used to inform decision making around resource allocation and intervention decisions e.g., hospital beds or stay-at-home orders. State of the art deep learning models often use multimodal data such as mobility or socio-demographic data to enhance COVID-19 case prediction models. Nevertheless, related work has revealed under-reporting bias in COVID-19 cases as well as sampling bias in mobility data for certain minority racial and ethnic groups, which could in turn affect the fairness of the COVID-19 predictions along race labels. In this paper, we show that state of the art deep learning models output mean prediction errors that are significantly different across racial and ethnic groups; and which could, in turn, support unfair policy decisions. We also propose a novel de-biasing method, DemOpts, to increase the fairness of deep learning based forecasting models trained on potentially biased datasets. Our results show that DemOpts can achieve better error parity that other state of the art de-biasing approaches, thus effectively reducing the differences in the mean error distributions across more racial and ethnic groups.
Paper Structure (20 sections, 6 equations, 4 figures, 5 tables, 1 algorithm)

This paper contains 20 sections, 6 equations, 4 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Literature
  3. Deep learning based Forecasting models
  4. Bias in Mobility and COVID-19 Data
  5. Fairness Metrics and Fairness Corrections in Machine learning models
  6. DemOpts Method
  7. Step 1: Calculate Loss
  8. Step 2: Identify Dependencies between Prediction Errors and Race and Ethnicity
  9. Step 3: Adjust the Loss
  10. DemOpts Evaluation Protocol
  11. Predictive Model and De-Biasing Approaches
  12. Measuring Model Fairness
  13. DemOpts Over State-of-the-Art
  14. DemOpts Evaluation Results
  15. Datasets
  16. ...and 5 more sections

Figures (4)

  • Figure 1: Flow Diagram for the DemOpts method.
  • Figure 2: Counties with majority based labels.
  • Figure 3: (a) Case counts per 1000 population for all ethnicities and race. (b) Case counts per 1000 population using majority based labelling of counties. Note scale difference in y-axis.
  • Figure 4: Mobility for all ethnicity and races.