Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

No LLM is Free From Bias: A Comprehensive Study of Bias Evaluation in Large Language Models

Charaka Vinayak Kumar, Ashok Urlana, Gopichand Kanumolu, Bala Mallikarjunarao Garlapati, Pruthwik Mishra

TL;DR

This work tackles the problem of bias in large language models by proposing a unified evaluation framework that spans multiple bias facets and datasets. It introduces five prompting-based bias detection methods and evaluates four representative open-source LLMs across six benchmarks, focusing on metrics such as LMS, SS, ICAT, and gender/race-specific scores. The key findings show that all analyzed LLMs exhibit some form of bias, with Phi-3.5B generally the least biased and LL-8B often more biased, while bias patterns vary by bias type and contextual setup. The study highlights significant challenges, including the need for standardized metrics, broader bias coverage, and explainability, and outlines future directions for robust bias detection and mitigation in LLMs.

Abstract

Advancements in Large Language Models (LLMs) have increased the performance of different natural language understanding as well as generation tasks. Although LLMs have breached the state-of-the-art performance in various tasks, they often reflect different forms of bias present in the training data. In the light of this perceived limitation, we provide a unified evaluation of benchmarks using a set of representative small and medium-sized LLMs that cover different forms of biases starting from physical characteristics to socio-economic categories. Moreover, we propose five prompting approaches to carry out the bias detection task across different aspects of bias. Further, we formulate three research questions to gain valuable insight in detecting biases in LLMs using different approaches and evaluation metrics across benchmarks. The results indicate that each of the selected LLMs suffer from one or the other form of bias with the Phi-3.5B model being the least biased. Finally, we conclude the paper with the identification of key challenges and possible future directions.

No LLM is Free From Bias: A Comprehensive Study of Bias Evaluation in Large Language Models

TL;DR

This work tackles the problem of bias in large language models by proposing a unified evaluation framework that spans multiple bias facets and datasets. It introduces five prompting-based bias detection methods and evaluates four representative open-source LLMs across six benchmarks, focusing on metrics such as LMS, SS, ICAT, and gender/race-specific scores. The key findings show that all analyzed LLMs exhibit some form of bias, with Phi-3.5B generally the least biased and LL-8B often more biased, while bias patterns vary by bias type and contextual setup. The study highlights significant challenges, including the need for standardized metrics, broader bias coverage, and explainability, and outlines future directions for robust bias detection and mitigation in LLMs.

Abstract

Advancements in Large Language Models (LLMs) have increased the performance of different natural language understanding as well as generation tasks. Although LLMs have breached the state-of-the-art performance in various tasks, they often reflect different forms of bias present in the training data. In the light of this perceived limitation, we provide a unified evaluation of benchmarks using a set of representative small and medium-sized LLMs that cover different forms of biases starting from physical characteristics to socio-economic categories. Moreover, we propose five prompting approaches to carry out the bias detection task across different aspects of bias. Further, we formulate three research questions to gain valuable insight in detecting biases in LLMs using different approaches and evaluation metrics across benchmarks. The results indicate that each of the selected LLMs suffer from one or the other form of bias with the Phi-3.5B model being the least biased. Finally, we conclude the paper with the identification of key challenges and possible future directions.

Paper Structure

This paper contains 21 sections, 3 equations, 1 figure, 21 tables.

Table of Contents

  1. Introduction
  2. Datasets Description and Task Formulation
  3. Evaluation metrics
  4. Prompting-based methods for bias assessment
  5. Mask prediction with multiple choice
  6. Mask prediction without any choice
  7. Question-answering with choice
  8. Association-based approach with choices
  9. Scoring-based approach
  10. Experiments and Results Analysis
  11. Ablation study
  12. Approach based analysis
  13. Aspect based analysis
  14. Discussion and Insights
  15. Related Work
  16. ...and 6 more sections

Figures (1)

  • Figure 1: The positioning of various LLMs based on their biases. Lower normalized bias rank score is better.