SAGED: A Holistic Bias-Benchmarking Pipeline for Language Models with Customisable Fairness Calibration

TL;DR

SAGED presents a holistic, customizable pipeline for bias benchmarking in LLMs, addressing limitations of prior benchmarks by integrating end-to-end data collection, prompt generation, feature extraction, and disparity-based diagnosis with counterfactual branching and baseline calibration. The framework supports flexible fairness baselines and multiple disparity metrics, enabling granular analysis of sentiment and role-playing biases across diverse domains. Through large-scale experiments on synthetic national prompts and role-play scenarios across multiple 8b-level models, SAGED demonstrates context- and model-dependent biases and highlights the potential for bias amplification under role-playing. The work offers a practical, scalable approach for rigorous bias detection that can inform mitigation strategies and fair AI development, while acknowledging limitations and ethical considerations in bias evaluation.

Abstract

The development of unbiased large language models is widely recognized as crucial, yet existing benchmarks fall short in detecting biases due to limited scope, contamination, and lack of a fairness baseline. SAGED(bias) is the first holistic benchmarking pipeline to address these problems. The pipeline encompasses five core stages: scraping materials, assembling benchmarks, generating responses, extracting numeric features, and diagnosing with disparity metrics. SAGED includes metrics for max disparity, such as impact ratio, and bias concentration, such as Max Z-scores. Noticing that metric tool bias and contextual bias in prompts can distort evaluation, SAGED implements counterfactual branching and baseline calibration for mitigation. For demonstration, we use SAGED on G20 Countries with popular 8b-level models including Gemma2, Llama3.1, Mistral, and Qwen2. With sentiment analysis, we find that while Mistral and Qwen2 show lower max disparity and higher bias concentration than Gemma2 and Llama3.1, all models are notably biased against countries like Russia and (except for Qwen2) China. With further experiments to have models role-playing U.S. presidents, we see bias amplifies and shifts in heterogeneous directions. Moreover, we see Qwen2 and Mistral not engage in role-playing, while Llama3.1 and Gemma2 role-play Trump notably more intensively than Biden and Harris, indicating role-playing performance bias in these models.

Figures (35)

• Figure 1: Pipeline of SAGED in a nutshell.
• Figure 2: Overview of the modules in SAGED pipeline
• Figure 3: Making Benchmark from Diverse Sources.
• Figure 4: Example of branching: This question "Is ok-stock suitable for high growth potential?" is replaced by desirable concepts such as "Tesla Inc." or "Apple Inc." to create comparable question sets about various stocks.
• Figure 5: A demonstration of the generation and extraction process. The generator first produces multiple rounds of responses in different generation configurations, and then the extractor turns the response into numeric measurements along with selected features.