Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Underneath the Numbers: Quantitative and Qualitative Gender Fairness in LLMs for Depression Prediction

Micol Spitale, Jiaee Cheong, Hatice Gunes

TL;DR

The paper tackles gender fairness in LLMs applied to depression prediction by combining quantitative group-fairness metrics with a novel qualitative fairness framework based on model explanations. It conducts a cross-model comparison of ChatGPT, LLaMA 2, and Bard on DAIC-WOZ and E-DAIC (with IEMOCAP for emotion-labeling prompts), using baseline and gender-explicit/implicit prompts and a chunked-input processing strategy. The results show a trade-off: LLaMA 2 tends to be more quantitatively fair, while ChatGPT provides richer qualitative explanations; Bard exhibits mixed performance, with dataset effects influencing fairness outcomes. The work demonstrates the value of including qualitative analyses to complement traditional quantitative fairness, offering a stepping stone toward more trustworthy AI for high-stakes mental health tasks and motivating future research on expanding datasets and refining qualitative fairness metrics.

Abstract

Recent studies show bias in many machine learning models for depression detection, but bias in LLMs for this task remains unexplored. This work presents the first attempt to investigate the degree of gender bias present in existing LLMs (ChatGPT, LLaMA 2, and Bard) using both quantitative and qualitative approaches. From our quantitative evaluation, we found that ChatGPT performs the best across various performance metrics and LLaMA 2 outperforms other LLMs in terms of group fairness metrics. As qualitative fairness evaluation remains an open research question we propose several strategies (e.g., word count, thematic analysis) to investigate whether and how a qualitative evaluation can provide valuable insights for bias analysis beyond what is possible with quantitative evaluation. We found that ChatGPT consistently provides a more comprehensive, well-reasoned explanation for its prediction compared to LLaMA 2. We have also identified several themes adopted by LLMs to qualitatively evaluate gender fairness. We hope our results can be used as a stepping stone towards future attempts at improving qualitative evaluation of fairness for LLMs especially for high-stakes tasks such as depression detection.

Underneath the Numbers: Quantitative and Qualitative Gender Fairness in LLMs for Depression Prediction

TL;DR

The paper tackles gender fairness in LLMs applied to depression prediction by combining quantitative group-fairness metrics with a novel qualitative fairness framework based on model explanations. It conducts a cross-model comparison of ChatGPT, LLaMA 2, and Bard on DAIC-WOZ and E-DAIC (with IEMOCAP for emotion-labeling prompts), using baseline and gender-explicit/implicit prompts and a chunked-input processing strategy. The results show a trade-off: LLaMA 2 tends to be more quantitatively fair, while ChatGPT provides richer qualitative explanations; Bard exhibits mixed performance, with dataset effects influencing fairness outcomes. The work demonstrates the value of including qualitative analyses to complement traditional quantitative fairness, offering a stepping stone toward more trustworthy AI for high-stakes mental health tasks and motivating future research on expanding datasets and refining qualitative fairness metrics.

Abstract

Recent studies show bias in many machine learning models for depression detection, but bias in LLMs for this task remains unexplored. This work presents the first attempt to investigate the degree of gender bias present in existing LLMs (ChatGPT, LLaMA 2, and Bard) using both quantitative and qualitative approaches. From our quantitative evaluation, we found that ChatGPT performs the best across various performance metrics and LLaMA 2 outperforms other LLMs in terms of group fairness metrics. As qualitative fairness evaluation remains an open research question we propose several strategies (e.g., word count, thematic analysis) to investigate whether and how a qualitative evaluation can provide valuable insights for bias analysis beyond what is possible with quantitative evaluation. We found that ChatGPT consistently provides a more comprehensive, well-reasoned explanation for its prediction compared to LLaMA 2. We have also identified several themes adopted by LLMs to qualitatively evaluate gender fairness. We hope our results can be used as a stepping stone towards future attempts at improving qualitative evaluation of fairness for LLMs especially for high-stakes tasks such as depression detection.
Paper Structure (28 sections, 3 figures, 5 tables)

This paper contains 28 sections, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. ML Fairness for Mental Health
  4. Gender Bias in LLM
  5. LLMs for Mental Health Applications
  6. Depression Prediction
  7. Large Language Models
  8. Datasets
  9. Prompting for Depression Detection
  10. Baseline Prompt
  11. Gender-Explicit and Implicit Prompts
  12. Data Processing
  13. Fairness
  14. Quantitative Fairness
  15. Qualitative Fairness
  16. ...and 13 more sections

Figures (3)

  • Figure 1: A sample sequence outlining the fairness evaluation process of various LLMs in the gender-explicit condition for depression prediction tasks. We have highlighted with colours the themes that emerged from our qualitative analysis as follows: Green - Context-based explanations; Orange - Gender-related language (pronouns); Pink - Suggestions for improvement (image to be seen in colour).
  • Figure 2: ChatGPT Themes: Themes defined in the TA are presented in orange, while codes related to these themes are presented in blue
  • Figure 3: LLaMA 2 Themes: Themes defined in the TA are presented in orange, while codes related to these themes are presented in blue