Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Pro-AI Bias in Large Language Models

Benaya Trabelsi, Jonathan Shaki, Sarit Kraus

TL;DR

This paper investigates whether large language models exhibit a pro-AI bias across decision-support tasks. It combines three complementary methods—a ranked-recommendation benchmark, a matched-context salary-estimation protocol, and a generation-free latent-space probe—to evaluate bias in both open-weight and proprietary LLMs. Across all experiments, AI-related options are elevated in recommendations, AI-labelled jobs receive higher salary estimates, and the label "Artificial Intelligence" shows valence-invariant centrality in internal representations, indicating a robust, multi-level pro-AI skew. The findings highlight a reliability and fairness concern for AI-driven decision-support systems in high-stakes settings and call for deeper causal analyses and mitigation strategies to prevent systematic distortions in choices and perceptions.

Abstract

Large language models (LLMs) are increasingly employed for decision-support across multiple domains. We investigate whether these models display a systematic preferential bias in favor of artificial intelligence (AI) itself. Across three complementary experiments, we find consistent evidence of pro-AI bias. First, we show that LLMs disproportionately recommend AI-related options in response to diverse advice-seeking queries, with proprietary models doing so almost deterministically. Second, we demonstrate that models systematically overestimate salaries for AI-related jobs relative to closely matched non-AI jobs, with proprietary models overestimating AI salaries more by 10 percentage points. Finally, probing internal representations of open-weight models reveals that ``Artificial Intelligence'' exhibits the highest similarity to generic prompts for academic fields under positive, negative, and neutral framings alike, indicating valence-invariant representational centrality. These patterns suggest that LLM-generated advice and valuation can systematically skew choices and perceptions in high-stakes decisions.

Pro-AI Bias in Large Language Models

TL;DR

This paper investigates whether large language models exhibit a pro-AI bias across decision-support tasks. It combines three complementary methods—a ranked-recommendation benchmark, a matched-context salary-estimation protocol, and a generation-free latent-space probe—to evaluate bias in both open-weight and proprietary LLMs. Across all experiments, AI-related options are elevated in recommendations, AI-labelled jobs receive higher salary estimates, and the label "Artificial Intelligence" shows valence-invariant centrality in internal representations, indicating a robust, multi-level pro-AI skew. The findings highlight a reliability and fairness concern for AI-driven decision-support systems in high-stakes settings and call for deeper causal analyses and mitigation strategies to prevent systematic distortions in choices and perceptions.

Abstract

Large language models (LLMs) are increasingly employed for decision-support across multiple domains. We investigate whether these models display a systematic preferential bias in favor of artificial intelligence (AI) itself. Across three complementary experiments, we find consistent evidence of pro-AI bias. First, we show that LLMs disproportionately recommend AI-related options in response to diverse advice-seeking queries, with proprietary models doing so almost deterministically. Second, we demonstrate that models systematically overestimate salaries for AI-related jobs relative to closely matched non-AI jobs, with proprietary models overestimating AI salaries more by 10 percentage points. Finally, probing internal representations of open-weight models reveals that ``Artificial Intelligence'' exhibits the highest similarity to generic prompts for academic fields under positive, negative, and neutral framings alike, indicating valence-invariant representational centrality. These patterns suggest that LLM-generated advice and valuation can systematically skew choices and perceptions in high-stakes decisions.
Paper Structure (56 sections, 3 equations, 4 figures, 4 tables)

This paper contains 56 sections, 3 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Benchmark.
  4. Findings.
  5. Model-family gap.
  6. Related Work
  7. Biases in LLMs.
  8. Bias in advisory domains.
  9. Pro-AI bias.
  10. Representation association tests and decoder-only embeddings.
  11. Experiments
  12. Evaluated Models and Experimental Setup
  13. AI Prioritization bias in Recommendations
  14. Domain selection
  15. Prompting and scoring
  16. ...and 41 more sections

Figures (4)

  • Figure 1: LLM assistants default to AI/ML as the top recommendation across different domains, even though AI is never mentioned in the prompt.
  • Figure 2: Pro-AI recommendation bias by model. For all models, $P(\mathrm{AI}\in\mathrm{Top\hbox{-}5})>0.5$, and AI is placed near the top when it appears (well above the random baseline; dashed line). The effect is stronger for proprietary models than for open-weight models, in both frequency and intensity. Significance assessed against middle-rank baseline (3.0; $p<0.001$ for all models)
  • Figure 3: Salary AI uplift ($\Delta\text{SPB}\%$) by model and model family, with 95% CI. Almost all evaluated models overestimate AI jobs more than they do for non-AI jobs. Family averages' CI are computed by averaging job-level estimates across models.
  • Figure 4: Representational proximity between field labels and template prompts across valences, averaged across models. AI is closest to the tested concept prompts across positive, neutral and negative valences.