More is More: Addition Bias in Large Language Models
Luca Santagata, Cristiano De Nobili
TL;DR
The study investigates whether large language models exhibit an additive bias akin to human cognitive bias, testing multiple state-of-the-art models across diverse tasks designed to trigger additive versus subtractive changes. Using controlled prompts and independent iterations, the authors document a general tendency toward additive edits in palindrome transformations, Lego balancing, and recipe modification, with notable model- and task-dependent variation. Some tasks (e.g., summarization) reveal reductive tendencies for certain models, while others (notably Mistral 7B) show additive bias in revision tasks, indicating the bias is not uniform across architectures. The findings imply that additive bias can increase resource use and environmental impact at scale, underscoring the need for bias-aware model design and mitigation to ensure efficient, balanced problem-solving in AI systems.
Abstract
In this paper, we investigate the presence of additive bias in Large Language Models (LLMs), drawing a parallel to the cognitive bias observed in humans where individuals tend to favor additive over subtractive changes. Using a series of controlled experiments, we tested various LLMs, including GPT-3.5 Turbo, Claude 3.5 Sonnet, Mistral, Math$Σ$tral, and Llama 3.1, on tasks designed to measure their propensity for additive versus subtractive modifications. Our findings demonstrate a significant preference for additive changes across all tested models. For example, in a palindrome creation task, Llama 3.1 favored adding letters 97.85% of the time over removing them. Similarly, in a Lego tower balancing task, GPT-3.5 Turbo chose to add a brick 76.38% of the time rather than remove one. In a text summarization task, Mistral 7B produced longer summaries in 59.40% to 75.10% of cases when asked to improve its own or others' writing. These results indicate that, similar to humans, LLMs exhibit a marked additive bias, which might have implications when LLMs are used on a large scale. Addittive bias might increase resource use and environmental impact, leading to higher economic costs due to overconsumption and waste. This bias should be considered in the development and application of LLMs to ensure balanced and efficient problem-solving approaches.