Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Decoy Effect in Search Interaction: A Pilot Study

Nuo Chen, Jiqun Liu, Tetsuya Sakai, Xiao-Ming Wu

TL;DR

The paper addresses how the decoy effect influences user interactions on Search Engine Result Pages (SERPs) by analyzing document-level decoys in two controlled datasets (THUIR2016 and THU-KDD). It defines decoys using similarity and quality thresholds and constructs control documents, then analyzes interaction signals with a regression framework that includes rank, task, and user controls. Logistic regression is used for click likelihood and OLS for duration and usefulness, revealing that decoys significantly increase click probability ($p<0.05$) and usefulness scores ($p<0.01$), with a non-significant trend towards longer browsing duration. This work is among the first to quantify decoy effects on SERP interactions, offering behavioral economics grounding for IR evaluation and highlighting future research directions on task themes, cognitive states, and fairness implications.

Abstract

In recent years, the influence of cognitive effects and biases on users' thinking, behaving, and decision-making has garnered increasing attention in the field of interactive information retrieval. The decoy effect, one of the main empirically confirmed cognitive biases, refers to the shift in preference between two choices when a third option (the decoy) which is inferior to one of the initial choices is introduced. However, it is not clear how the decoy effect influences user interactions with and evaluations on Search Engine Result Pages (SERPs). To bridge this gap, our study seeks to understand how the decoy effect at the document level influences users' interaction behaviors on SERPs, such as clicks, dwell time, and usefulness perceptions. We conducted experiments on two publicly available user behavior datasets and the findings reveal that, compared to cases where no decoy is present, the probability of a document being clicked could be improved and its usefulness score could be higher, should there be a decoy associated with the document.

Decoy Effect in Search Interaction: A Pilot Study

TL;DR

The paper addresses how the decoy effect influences user interactions on Search Engine Result Pages (SERPs) by analyzing document-level decoys in two controlled datasets (THUIR2016 and THU-KDD). It defines decoys using similarity and quality thresholds and constructs control documents, then analyzes interaction signals with a regression framework that includes rank, task, and user controls. Logistic regression is used for click likelihood and OLS for duration and usefulness, revealing that decoys significantly increase click probability () and usefulness scores (), with a non-significant trend towards longer browsing duration. This work is among the first to quantify decoy effects on SERP interactions, offering behavioral economics grounding for IR evaluation and highlighting future research directions on task themes, cognitive states, and fairness implications.

Abstract

In recent years, the influence of cognitive effects and biases on users' thinking, behaving, and decision-making has garnered increasing attention in the field of interactive information retrieval. The decoy effect, one of the main empirically confirmed cognitive biases, refers to the shift in preference between two choices when a third option (the decoy) which is inferior to one of the initial choices is introduced. However, it is not clear how the decoy effect influences user interactions with and evaluations on Search Engine Result Pages (SERPs). To bridge this gap, our study seeks to understand how the decoy effect at the document level influences users' interaction behaviors on SERPs, such as clicks, dwell time, and usefulness perceptions. We conducted experiments on two publicly available user behavior datasets and the findings reveal that, compared to cases where no decoy is present, the probability of a document being clicked could be improved and its usefulness score could be higher, should there be a decoy associated with the document.
Paper Structure (8 sections, 4 figures, 1 table)

This paper contains 8 sections, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Research Question
  4. Experiment
  5. Datasets
  6. Data Processing Approach
  7. Data Analysis and Experimental Results
  8. Conclusion

Figures (4)

  • Figure 1: An example of the decoy effect. The image is generated by the authors using Midjourney.
  • Figure 2: Data processing flow in the first experiment.
  • Figure 3: The distribution of click probability, browsing time and usefulness score on THUIR2016 dataset (top) and THU-KDD (bottom) dataset respectively.
  • Figure 4: The distribution of rank on THUIR2016 (left) dataset and THU-KDD (right) dataset respectively.