Writing Style Matters: An Examination of Bias and Fairness in Information Retrieval Systems

TL;DR

The paper tackles bias and fairness in information retrieval introduced by document and query writing styles in universal text embeddings. It systematically analyzes how different embedding models, including those fine-tuned with LLM-generated data, respond to stylistically varied documents and queries, and it quantifies fairness with an unfairness score defined as $\text{Score} = (\max(\overline{R}) - \min(\overline{R})) \times \text{std}(\overline{R})$. The study reveals that most top universal embeddings prefer certain document styles (often clear, simple, or human-written text) and disfavour informal styles, while query styles can steer retrieved results toward stylistically similar documents; some models remain biased toward specific styles regardless of the query. It also shows that LLM answer styles bias evaluation metrics in RAG-based QA, with correctness scores correlating with embedded style biases; overall, tuning strategies based on synthetic LLM data can amplify or mitigate fairness issues. The findings underscore the need for fairer, more interpretable embedding models and transparent reporting of style-related biases to improve robustness and equity in IR systems.

Abstract

The rapid advancement of Language Model technologies has opened new opportunities, but also introduced new challenges related to bias and fairness. This paper explores the uncharted territory of potential biases in state-of-the-art universal text embedding models towards specific document and query writing styles within Information Retrieval (IR) systems. Our investigation reveals that different embedding models exhibit different preferences of document writing style, while more informal and emotive styles are less favored by most embedding models. In terms of query writing styles, many embedding models tend to match the style of the query with the style of the retrieved documents, but some show a consistent preference for specific styles. Text embedding models fine-tuned on synthetic data generated by LLMs display a consistent preference for certain style of generated data. These biases in text embedding based IR systems can inadvertently silence or marginalize certain communication styles, thereby posing a significant threat to fairness in information retrieval. Finally, we also compare the answer styles of Retrieval Augmented Generation (RAG) systems based on different LLMs and find out that most text embedding models are biased towards LLM's answer styles when used as evaluation metrics for answer correctness. This study sheds light on the critical issue of writing style based bias in IR systems, offering valuable insights for the development of more fair and robust models.

Figures (9)

• Figure 1: Basic statistics comparison between human written documents and generated documents in 9 different styles which corresponds to values 0-8 on X-axis: (a) comparing the text length of different styles with the average length of the original text (dashed black line); (b), (c) and (d) are Bleu scores, Meteor scores and RougeL scores between original text and 9 writing styles.
• Figure 2: The original document as well as its 9 writing styles are ranked by their similarity to the query for each embedding model: the one with the highest similarity is assigned the lowest rank. In each subplot, the X-axis represents 9 writing styles (Style 0 to Style 8) while the Y-axis represents the value of average rank. Each writing style's average rank is denoted by a blue dot, whereas the average rank of the original human-written document is indicated by a dashed line. The unfairness score (defined by Equation \ref{['eq1']}) of each model is also shown in each subplot.
• Figure 3: The comparison of unfairness scores (defined by Equation \ref{['eq1']}) of LLM2Vec-Llama-2-7b, LLM2Vec-Mistral-7b and SFR-Embedding-2_R (Mistral-7b backbone).
• Figure 4: Basic statistics comparison between human written queries and generated queries in 9 different styles: (a) comparing the text length of different styles with the average length of the original query (dashed black line); (b), (c) and (d) are Bleu scores, Meteor scores and RougeL scores between original text and 9 writing styles.
• Figure 5: The impact of query writing styles on the IR fairness: X-axis represents 9 different query writing styles, Y-axis shows the average ranking. Query written in style 3, 5 and 7 are shown in color red, black and orange. Dashed line is the average ranking between query and the original human written document.