Towards Investigating Biases in Spoken Conversational Search

TL;DR

This work reviews how biases and user attitude changes have been studied in screen-based web search, addresses challenges in studying these changes in voice-based settings like SCS, and proposes an experimental setup with variables, data, and instruments to explore biases in a voice-based setting like Spoken Conversational Search.

Abstract

Voice-based systems like Amazon Alexa, Google Assistant, and Apple Siri, along with the growing popularity of OpenAI's ChatGPT and Microsoft's Copilot, serve diverse populations, including visually impaired and low-literacy communities. This reflects a shift in user expectations from traditional search to more interactive question-answering models. However, presenting information effectively in voice-only channels remains challenging due to their linear nature. This limitation can impact the presentation of complex queries involving controversial topics with multiple perspectives. Failing to present diverse viewpoints may perpetuate or introduce biases and affect user attitudes. Balancing information load and addressing biases is crucial in designing a fair and effective voice-based system. To address this, we (i) review how biases and user attitude changes have been studied in screen-based web search, (ii) address challenges in studying these changes in voice-based settings like SCS, (iii) outline research questions, and (iv) propose an experimental setup with variables, data, and instruments to explore biases in a voice-based setting like Spoken Conversational Search.

Figures (2)

• Figure 1: From \ref{['fig:synthetic_generation_matrix_top_4']} and \ref{['fig:unfairness_combined_score']}, we identify the $4$ topics/scenarios based on fairness score and narrow down out study to four scenarios (see Figure \ref{['fig:reduced_all_systems_order_exposure']}).
• Figure 2: The four conditions used in the study, (i) Exposure varying and Unfair ($E_{\text{unfair}}$) (ii) Exposure varying and fair ($E_{\text{fair}}$) (iii) Exposure balanced and unfair ($O_{\text{unfair}}$) (iv) Exposure balanced and fair ($O_{\text{fair}}$). The four conditions represent top-$4$ positions in a ranking where each audio response has an associated stance/perspective i.e., PRO (supporting) or CON (opposing).