Bias in Text Embedding Models
Vasyl Rakivnenko, Nestor Maslej, Jessica Cervi, Volodymyr Zhukov
TL;DR
The paper investigates gender bias in widely used text embedding models by measuring how occupations are differentially associated with gendered terms across multiple providers. It uses a standardized benchmark derived from a BIG-bench dataset to compute bias-score differences for woman/man and girl/boy associations, comparing AI21, Amazon, BAAI, Cohere, Google, Meta, OpenAI, Voyage AI, and TU Darmstadt models. Results show that all models exhibit gender bias, with substantial variation in magnitude and direction across models and even within the same model under different prompts. The findings emphasize model-specific biases and prompt sensitivity, highlighting practical implications for businesses deploying embeddings and pointing to the need for broader bias analysis and mitigation in future work.
Abstract
Text embedding is becoming an increasingly popular AI methodology, especially among businesses, yet the potential of text embedding models to be biased is not well understood. This paper examines the degree to which a selection of popular text embedding models are biased, particularly along gendered dimensions. More specifically, this paper studies the degree to which these models associate a list of given professions with gendered terms. The analysis reveals that text embedding models are prone to gendered biases but in varying ways. Although there are certain inter-model commonalities, for instance, greater association of professions like nurse, homemaker, and socialite with female identifiers, and greater association of professions like CEO, manager, and boss with male identifiers, not all models make the same gendered associations for each occupation. Furthermore, the magnitude and directionality of bias can also vary on a model-by-model basis and depend on the particular words models are prompted with. This paper demonstrates that gender bias afflicts text embedding models and suggests that businesses using this technology need to be mindful of the specific dimensions of this problem.