Towards Detecting and Mitigating Cognitive Bias in Spoken Conversational Search

TL;DR

This paper confronts cognitive biases in Spoken Conversational Search (SCS), where voice-only interaction complicates bias detection. It proposes a multimodal framework leveraging wearable sensors and neural signals (e.g., EEG, EDA) to characterize bias across querying, consumption, and judgment stages, and to evaluate mitigation strategies such as clarifying questions and content nudges. A case study on Spoken Conversational Argumentative Search (SCAS) illustrates data sources and experimental design, while a lab pilot using EEG/EDA demonstrates detectable differences in cognitive load between easy and hard listening tasks. The work highlights substantial methodological, interpretability, and ethical challenges and argues for careful, preregistered, multimodal research to advance bias-aware SCS systems with real-world impact.

Abstract

Instruments such as eye-tracking devices have contributed to understanding how users interact with screen-based search engines. However, user-system interactions in audio-only channels -- as is the case for Spoken Conversational Search (SCS) -- are harder to characterize, given the lack of instruments to effectively and precisely capture interactions. Furthermore, in this era of information overload, cognitive bias can significantly impact how we seek and consume information -- especially in the context of controversial topics or multiple viewpoints. This paper draws upon insights from multiple disciplines (including information seeking, psychology, cognitive science, and wearable sensors) to provoke novel conversations in the community. To this end, we discuss future opportunities and propose a framework including multimodal instruments and methods for experimental designs and settings. We demonstrate preliminary results as an example. We also outline the challenges and offer suggestions for adopting this multimodal approach, including ethical considerations, to assist future researchers and practitioners in exploring cognitive biases in SCS.

Figures (6)

• Figure 1: Structure Diagram of this paper.
• Figure 2: Example interaction between a user and a SCAS system involving two perspectives, i.e., supporting (PRO) and opposing (CON).
• Figure 3: Wizard of Oz (WOZ) Set-up where the right side depicts a participant wearing sensors while interacting with the SCS system. The left side depicts the intermediary placed separately from the participant to simulate an information access system that provides a response based on the user queries.
• Figure 4: 10--20 System for EEG electrode locations and corresponding brain regions MARTINEZMALDONADO202046bos2006eeg, which could help to characterize the type of brain activities (and corresponding physiology, e.g., cognitive, emotional, and behavioral states).
• Figure 5: Preliminary EEG Results ($N=7$) of grand average on listening to Search Results on self-rated $easy$ (Antarctica exploration -- R03.353) and $hard$ topics (Freighter ship registration -- T04.743). Deeper color indicates greater neural activities. Cool colors for negative voltage represent inhibitory activities, i.e., suppressed, blocked, or restricted, while warm colors for positive voltage represent excitatory activities, i.e., promoted, facilitated, or enhanced luck_kappenman_2016. The dots represent the placement of 14 electrodes.