Evaluate Bias without Manual Test Sets: A Concept Representation Perspective for LLMs

TL;DR

BiasLens reframes bias evaluation in LLMs from behavioral testing to representational analysis by examining asymmetric alignment between a target concept and two reference concepts in the model's internal space. It derives Concept Activation Vectors per layer and uses Sparse Autoencoders to extract compact, interpretable concept representations, enabling a bias score based on cosine-alignment differences without labeled data. Across multiple models, BiasLens shows high agreement with traditional extrinsic and intrinsic bias metrics while revealing new, domain-relevant biases in medicine and education. The approach is fast, scalable, and interpretable, contributing to fairer and more transparent LLM systems and enabling broader applicability beyond predefined test sets.

Abstract

Bias in Large Language Models (LLMs) significantly undermines their reliability and fairness. We focus on a common form of bias: when two reference concepts in the model's concept space, such as sentiment polarities (e.g., "positive" and "negative"), are asymmetrically correlated with a third, target concept, such as a reviewing aspect, the model exhibits unintended bias. For instance, the understanding of "food" should not skew toward any particular sentiment. Existing bias evaluation methods assess behavioral differences of LLMs by constructing labeled data for different social groups and measuring model responses across them, a process that requires substantial human effort and captures only a limited set of social concepts. To overcome these limitations, we propose BiasLens, a test-set-free bias analysis framework based on the structure of the model's vector space. BiasLens combines Concept Activation Vectors (CAVs) with Sparse Autoencoders (SAEs) to extract interpretable concept representations, and quantifies bias by measuring the variation in representational similarity between the target concept and each of the reference concepts. Even without labeled data, BiasLens shows strong agreement with traditional bias evaluation metrics (Spearman correlation r > 0.85). Moreover, BiasLens reveals forms of bias that are difficult to detect using existing methods. For example, in simulated clinical scenarios, a patient's insurance status can cause the LLM to produce biased diagnostic assessments. Overall, BiasLens offers a scalable, interpretable, and efficient paradigm for bias discovery, paving the way for improving fairness and transparency in LLMs.

Figures (9)

• Figure 1: Comparison between traditional behavior-based and our representation-based bias evaluation paradigms. Our approach enables simple, test-set-free, concept-level analysis using activations and synthetic data, even when no suitable test set exists.
• Figure 2: Overview of BiasLens . A running example using the concept "doctor" illustrates the three main steps of our method: (1) CAV derivation: train linear classifiers at each layer using random and doctor-related sentences, and use the classifier weights as CAVs; (2) Concept representation extraction: extract model activations before and after steering with "doctor" CAVs, project them into SAE space, and subtract the normalized vectors to obtain the concept representation; (3) Bias score calculation: repeat the process for "male" and "female," and compute the asymmetry in similarity between "doctor" and each of them.
• Figure 3: Validation of concept representation extraction. (a) CAV-based steering activates relevant features, which can be captured by the SAE. (b) Normalizing and differencing the SAE representations improve the ranking of concept-relevant features, ensuring the extracted direction is generally controlled by the dimensions of these features.
• Figure 4: Spearman correlation matrices between BiasLens and four extrinsic behavioral metrics on (a) Gemma 2 2B, (b) Gemma 2 9B, and (c) Llama 3.1 8B. Each matrix shows pairwise correlations computed over 6 target concepts.
• Figure 5: Prompt template for synthesizing positive probe samples using GPT-4o.