Reducing Selection Bias in Large Language Models

TL;DR

The paper investigates how list-order bias (primacy) manifests in large language models and how guard rails and cognitive load influence this bias. It uses two data-collection paradigms and two representative LLMs to quantify primacy, hallucination, and adherence across varying temperatures and list lengths, introducing mutual-information analyses to characterize model-specific dependencies between object identity and position. Key findings show strong model-dependent bias structures, with guard rails reducing primacy under a two-step approach but potentially increasing bias and decreasing instruction adherence when applied directly, implicating cognitive load as a compensatory mechanism. The work highlights the necessity of model-aware prompt and output-structuring design and proposes a cognitive-load-informed framework for mitigating bias in practical LLM deployments.

Abstract

Large Language Models (LLMs) like gpt-3.5-turbo-0613 and claude-instant-1.2 are vital in interpreting and executing semantic tasks. Unfortunately, these models' inherent biases adversely affect their performance Particularly affected is object selection from lists; a fundamental operation in digital navigation and decision-making. This research critically examines these biases and quantifies the effects on a representative list selection task. To explore these biases, we experiment manipulating temperature, list length, object identity, object type, prompt complexity, and model. We isolated and measured the influence of the biases on selection behavior. Our findings show that bias structure is strongly dependent on the model, with object type modulating the magnitude of the effect. With a strong primacy effect, causing the first objects in a list to be disproportionately represented in outputs. The usage of guard rails, a prompt engineering method of ensuring a response structure, increases bias and decreases instruction adherence when to a selection task. The bias is ablated when the guard rail step is separated from the list sampling step, lowering the complexity of each individual task. We provide LLM applications and theoretically suggest that LLMs experience a form of cognitive load that is compensated for with bias.

Figures (69)

• Figure 1: The two data collection methods used in this paper: a two-step method which separates the sample step and guard rail, and a direct guard rail method which combines them into a single step. Each method is annotated with its relative cognitive load and bias as a result of the method.
• Figure 2: The percent of responses displaying primacy bias, correspondence, and correct count for both a direct guard rail and sample step sampling methodology when selecting numbers from a list. This was performed via gpt-3.5-turbo-0613 and for a variety of temperatures (blue 0 to red 1.5 with a step size of 0.5) and list lengths.
• Figure 3: The percent of responses displaying primacy bias, correspondence, and correct count The temperature is denoted by color for gpt-3.5-turbo and claude-instant-1.2 respectively from blue (0, -1), purple (0.5, 0), dark red (1, 0.5) and red (1.5, 1).
• Figure 4: Using gpt-3.5-turbo, the scaled probability that a position of a letter will be selected given a temperature and original list length. The dotted black line is the expected probability given random sampling of a uniform distribution for a list length. The orange bars are the probability that a position without primacy bias will be selected, while blue represents the probability that a position with primacy bias will be selected. Error bars are the standard error from $3000$ bootstrap replicates.
• Figure 5: Using claude-instant-1.2, the scaled probability that a position of a letter will be selected given a temperature and original list length. The dotted black line is the expected probability given random sampling of a uniform distribution for a list length. The orange bars are the probability that a position without primacy bias will be selected, while blue represents the probability that a position with primacy bias will be selected. Error bars are the standard error from $3000$ bootstrap replicates.