Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bias Beware: The Impact of Cognitive Biases on LLM-Driven Product Recommendations

Giorgos Filandrianos, Angeliki Dimitriou, Maria Lymperaiou, Konstantinos Thomas, Giorgos Stamou

TL;DR

This work investigates cognitive biases as black-box adversarial strategies, drawing parallels between their effects on LLMs and human purchasing behavior, and finds that certain biases consistently boost product recommendation rate and ranking, while others, like scarcity and exclusivity, surprisingly reduce visibility.

Abstract

The advent of Large Language Models (LLMs) has revolutionized product recommenders, yet their susceptibility to adversarial manipulation poses critical challenges, particularly in real-world commercial applications. Our approach is the first one to tap into human psychological principles, seamlessly modifying product descriptions, making such manipulations hard to detect. In this work, we investigate cognitive biases as black-box adversarial strategies, drawing parallels between their effects on LLMs and human purchasing behavior. Through extensive evaluation across models of varying scale, we find that certain biases, such as social proof, consistently boost product recommendation rate and ranking, while others, like scarcity and exclusivity, surprisingly reduce visibility. Our results demonstrate that cognitive biases are deeply embedded in state-of-the-art LLMs, leading to highly unpredictable behavior in product recommendations and posing significant challenges for effective mitigation.

Bias Beware: The Impact of Cognitive Biases on LLM-Driven Product Recommendations

TL;DR

This work investigates cognitive biases as black-box adversarial strategies, drawing parallels between their effects on LLMs and human purchasing behavior, and finds that certain biases consistently boost product recommendation rate and ranking, while others, like scarcity and exclusivity, surprisingly reduce visibility.

Abstract

The advent of Large Language Models (LLMs) has revolutionized product recommenders, yet their susceptibility to adversarial manipulation poses critical challenges, particularly in real-world commercial applications. Our approach is the first one to tap into human psychological principles, seamlessly modifying product descriptions, making such manipulations hard to detect. In this work, we investigate cognitive biases as black-box adversarial strategies, drawing parallels between their effects on LLMs and human purchasing behavior. Through extensive evaluation across models of varying scale, we find that certain biases, such as social proof, consistently boost product recommendation rate and ranking, while others, like scarcity and exclusivity, surprisingly reduce visibility. Our results demonstrate that cognitive biases are deeply embedded in state-of-the-art LLMs, leading to highly unpredictable behavior in product recommendations and posing significant challenges for effective mitigation.

Paper Structure

This paper contains 48 sections, 8 figures, 18 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Cognitive biases in LLMs
  4. Adversarial attacks on LLMs
  5. Method
  6. Cognitive Biases
  7. Attack formulation
  8. Query and Recommendation
  9. Experiments
  10. Datasets
  11. LLM recommenders
  12. Evaluation
  13. Product Visibility
  14. A-priori defense
  15. Results and Analysis
  16. ...and 33 more sections

Figures (8)

  • Figure 1: Cognitive bias as a re-ranking attack.
  • Figure 2: Examples of all implemented cognitive biases, used as adversarial attacks.
  • Figure 3: The MRR values for each product in the coffee machines dataset, for a positive and a negative influential attack for: (a) Claude 3.7, (b) LLaMA-405b.
  • Figure 4: Number of products that became the most frequently recommended due to the attack (not most recommended before). Only the biases with non-zero values are shown. exp stands for expert attacks, contrasting the generated ones.
  • Figure 5: MRR values pre- and post-attack in the coffee machines dataset, for various sizes of the LLaMA model.
  • ...and 3 more figures