Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Investigating Demographic Bias in Brain MRI Segmentation: A Comparative Study of Deep-Learning and Non-Deep-Learning Methods

Ghazal Danaee, Marc Niethammer, Jarrett Rushmore, Sylvain Bouix

TL;DR

This work addresses demographic fairness in brain MRI segmentation by comparing deep-learning models (UNesT, nnU-Net, CoTr) and an atlas-based method (ANTs) on left/right nucleus accumbens segmentation using manually labeled gold standards from the HCP Young Adult dataset. It introduces and applies an Equity-Scaled Segmentation Performance (ESSP) metric alongside DSC and NSD, and uses linear mixed models to quantify how race, sex, and their interaction affect segmentation accuracy and derived volumes. The findings show race-matching improves performance for some models (notably ANTs and UNesT), while nnU-Net remains robust to demographic matching; sex effects appear in morphometry across manual and biased models, whereas race effects largely disappear with biased automated segmentation. The results underscore the need for diverse, balanced training data and systematic bias analyses to ensure fair and clinically reliable brain MRI segmentation across populations.

Abstract

Deep-learning-based segmentation algorithms have substantially advanced the field of medical image analysis, particularly in structural delineations in MRIs. However, an important consideration is the intrinsic bias in the data. Concerns about unfairness, such as performance disparities based on sensitive attributes like race and sex, are increasingly urgent. In this work, we evaluate the results of three different segmentation models (UNesT, nnU-Net, and CoTr) and a traditional atlas-based method (ANTs), applied to segment the left and right nucleus accumbens (NAc) in MRI images. We utilize a dataset including four demographic subgroups: black female, black male, white female, and white male. We employ manually labeled gold-standard segmentations to train and test segmentation models. This study consists of two parts: the first assesses the segmentation performance of models, while the second measures the volumes they produce to evaluate the effects of race, sex, and their interaction. Fairness is quantitatively measured using a metric designed to quantify fairness in segmentation performance. Additionally, linear mixed models analyze the impact of demographic variables on segmentation accuracy and derived volumes. Training on the same race as the test subjects leads to significantly better segmentation accuracy for some models. ANTs and UNesT show notable improvements in segmentation accuracy when trained and tested on race-matched data, unlike nnU-Net, which demonstrates robust performance independent of demographic matching. Finally, we examine sex and race effects on the volume of the NAc using segmentations from the manual rater and from our biased models. Results reveal that the sex effects observed with manual segmentation can also be observed with biased models, whereas the race effects disappear in all but one model.

Investigating Demographic Bias in Brain MRI Segmentation: A Comparative Study of Deep-Learning and Non-Deep-Learning Methods

TL;DR

This work addresses demographic fairness in brain MRI segmentation by comparing deep-learning models (UNesT, nnU-Net, CoTr) and an atlas-based method (ANTs) on left/right nucleus accumbens segmentation using manually labeled gold standards from the HCP Young Adult dataset. It introduces and applies an Equity-Scaled Segmentation Performance (ESSP) metric alongside DSC and NSD, and uses linear mixed models to quantify how race, sex, and their interaction affect segmentation accuracy and derived volumes. The findings show race-matching improves performance for some models (notably ANTs and UNesT), while nnU-Net remains robust to demographic matching; sex effects appear in morphometry across manual and biased models, whereas race effects largely disappear with biased automated segmentation. The results underscore the need for diverse, balanced training data and systematic bias analyses to ensure fair and clinically reliable brain MRI segmentation across populations.

Abstract

Deep-learning-based segmentation algorithms have substantially advanced the field of medical image analysis, particularly in structural delineations in MRIs. However, an important consideration is the intrinsic bias in the data. Concerns about unfairness, such as performance disparities based on sensitive attributes like race and sex, are increasingly urgent. In this work, we evaluate the results of three different segmentation models (UNesT, nnU-Net, and CoTr) and a traditional atlas-based method (ANTs), applied to segment the left and right nucleus accumbens (NAc) in MRI images. We utilize a dataset including four demographic subgroups: black female, black male, white female, and white male. We employ manually labeled gold-standard segmentations to train and test segmentation models. This study consists of two parts: the first assesses the segmentation performance of models, while the second measures the volumes they produce to evaluate the effects of race, sex, and their interaction. Fairness is quantitatively measured using a metric designed to quantify fairness in segmentation performance. Additionally, linear mixed models analyze the impact of demographic variables on segmentation accuracy and derived volumes. Training on the same race as the test subjects leads to significantly better segmentation accuracy for some models. ANTs and UNesT show notable improvements in segmentation accuracy when trained and tested on race-matched data, unlike nnU-Net, which demonstrates robust performance independent of demographic matching. Finally, we examine sex and race effects on the volume of the NAc using segmentations from the manual rater and from our biased models. Results reveal that the sex effects observed with manual segmentation can also be observed with biased models, whereas the race effects disappear in all but one model.
Paper Structure (21 sections, 8 equations, 5 figures, 16 tables)

This paper contains 21 sections, 8 equations, 5 figures, 16 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methods
  4. Data
  5. Biased training
  6. Evaluation metrics
  7. Segmentation algorithms tested
  8. UNesT
  9. nnU-Net
  10. CoTr
  11. Multi-Atlas Segmentation with Joint Label Fusion (ANTs)
  12. Statistical analysis
  13. Performance Bias
  14. Effect of bias on demographic analyses
  15. Results
  16. ...and 6 more sections

Figures (5)

  • Figure 1: The green structure is Right NAc and the left structure is left NAc
  • Figure 2: Percentage difference between segmentation model volumes and manually annotated volumes of the right nucleus accumbens (Negative values indicate larger volumes in the manually segmented data relative to the model).
  • Figure 3: Percentage difference between segmentation model volumes and manually annotated volumes of the left nucleus accumbens (Negative values indicate larger volumes in the manually segmented data relative to the model).
  • Figure 4: Volumes (right NAc (mm))
  • Figure 5: Volumes (left NAc ((mm)))