Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Perception-Aware Bias Detection for Query Suggestions

Fabian Haak, Philipp Schaer

TL;DR

Bias detection in query suggestions, particularly for person-related searches, is challenging due to sparse context and brief exposure. The paper extends the Bonart et al. pipeline by embedding perception-aware metrics based on $DCG$ and $nDCG$ to account for rank- and frequency-aware topical exposure, and validates the approach on a German politicians dataset. Results reveal significant gender- and age-related biases that are more detectable with the perception-aware metrics than with simple counts, highlighting the importance of ranking and exposure in user perception. This work improves bias-detection fidelity in query suggestions and provides a framework adaptable to other ranked, sparse linguistic contexts.

Abstract

Bias in web search has been in the spotlight of bias detection research for quite a while. At the same time, little attention has been paid to query suggestions in this regard. Awareness of the problem of biased query suggestions has been raised. Likewise, there is a rising need for automatic bias detection approaches. This paper adds on the bias detection pipeline for bias detection in query suggestions of person-related search developed by Bonart et al. \cite{Bonart_2019a}. The sparseness and lack of contextual metadata of query suggestions make them a difficult subject for bias detection. Furthermore, query suggestions are perceived very briefly and subliminally. To overcome these issues, perception-aware metrics are introduced. Consequently, the enhanced pipeline is able to better detect systematic topical bias in search engine query suggestions for person-related searches. The results of an analysis performed with the developed pipeline confirm this assumption. Due to the perception-aware bias detection metrics, findings produced by the pipeline can be assumed to reflect bias that users would discern.

Perception-Aware Bias Detection for Query Suggestions

TL;DR

Bias detection in query suggestions, particularly for person-related searches, is challenging due to sparse context and brief exposure. The paper extends the Bonart et al. pipeline by embedding perception-aware metrics based on and to account for rank- and frequency-aware topical exposure, and validates the approach on a German politicians dataset. Results reveal significant gender- and age-related biases that are more detectable with the perception-aware metrics than with simple counts, highlighting the importance of ranking and exposure in user perception. This work improves bias-detection fidelity in query suggestions and provides a framework adaptable to other ranked, sparse linguistic contexts.

Abstract

Bias in web search has been in the spotlight of bias detection research for quite a while. At the same time, little attention has been paid to query suggestions in this regard. Awareness of the problem of biased query suggestions has been raised. Likewise, there is a rising need for automatic bias detection approaches. This paper adds on the bias detection pipeline for bias detection in query suggestions of person-related search developed by Bonart et al. \cite{Bonart_2019a}. The sparseness and lack of contextual metadata of query suggestions make them a difficult subject for bias detection. Furthermore, query suggestions are perceived very briefly and subliminally. To overcome these issues, perception-aware metrics are introduced. Consequently, the enhanced pipeline is able to better detect systematic topical bias in search engine query suggestions for person-related searches. The results of an analysis performed with the developed pipeline confirm this assumption. Due to the perception-aware bias detection metrics, findings produced by the pipeline can be assumed to reflect bias that users would discern.
Paper Structure (13 sections, 3 equations, 2 figures, 2 tables)

This paper contains 13 sections, 3 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Data Acquisition.
  5. Preprocessing.
  6. Bias Analysis.
  7. Regression Analysis.
  8. Results of Bias Analysis of German Politicians Dataset
  9. Data Acquisition.
  10. Preprocessing.
  11. Bias Analysis.
  12. Discussion
  13. Conclusion

Figures (2)

  • Figure 1: Bias detection pipeline for person-related query suggestions developed by Bonart et al. Bonart_2019a. The highlighted steps have been modified.
  • Figure 2: DCG and nDCG scores as well as total appearance percentages for gender and age meta-attributes. The dataset includes 818 male and 420 female politicians, 1096 older than or exactly 40 years old and 1414 younger than 40 years.