Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Auditing Search Query Suggestion Bias Through Recursive Algorithm Interrogation

Fabian Haak, Philipp Schaer

TL;DR

This paper tackles bias in search query suggestions within political domains by addressing data sparsity with Recursive Algorithm Interrogation (RAI) to build suggestion trees that capture deeper, subliminal queries beyond top-level suggestions. It combines RAI-driven data collection, Word2Vec-based semantic vectorization, and three-cluster topical analysis with regression to detect group biases across gender, age, party, and role among German politicians. The approach yields a large, more complete dataset and reveals nuanced, often small, bias signals—most notably gender-related differences in cluster distributions and role-based shifts in topical exposure—challenging some prior findings that relied on more limited data. The method demonstrates the potential of complete suggestion trees for bias auditing and suggests avenues to enhance validity through perception-aware metrics, broader engines, and domain expansion, with practical implications for understanding how suggestion systems shape political information consumption.

Abstract

Despite their important role in online information search, search query suggestions have not been researched as much as most other aspects of search engines. Although reasons for this are multi-faceted, the sparseness of context and the limited data basis of up to ten suggestions per search query pose the most significant problem in identifying bias in search query suggestions. The most proven method to reduce sparseness and improve the validity of bias identification of search query suggestions so far is to consider suggestions from subsequent searches over time for the same query. This work presents a new, alternative approach to search query bias identification that includes less high-level suggestions to deepen the data basis of bias analyses. We employ recursive algorithm interrogation techniques and create suggestion trees that enable access to more subliminal search query suggestions. Based on these suggestions, we investigate topical group bias in person-related searches in the political domain.

Auditing Search Query Suggestion Bias Through Recursive Algorithm Interrogation

TL;DR

This paper tackles bias in search query suggestions within political domains by addressing data sparsity with Recursive Algorithm Interrogation (RAI) to build suggestion trees that capture deeper, subliminal queries beyond top-level suggestions. It combines RAI-driven data collection, Word2Vec-based semantic vectorization, and three-cluster topical analysis with regression to detect group biases across gender, age, party, and role among German politicians. The approach yields a large, more complete dataset and reveals nuanced, often small, bias signals—most notably gender-related differences in cluster distributions and role-based shifts in topical exposure—challenging some prior findings that relied on more limited data. The method demonstrates the potential of complete suggestion trees for bias auditing and suggests avenues to enhance validity through perception-aware metrics, broader engines, and domain expansion, with practical implications for understanding how suggestion systems shape political information consumption.

Abstract

Despite their important role in online information search, search query suggestions have not been researched as much as most other aspects of search engines. Although reasons for this are multi-faceted, the sparseness of context and the limited data basis of up to ten suggestions per search query pose the most significant problem in identifying bias in search query suggestions. The most proven method to reduce sparseness and improve the validity of bias identification of search query suggestions so far is to consider suggestions from subsequent searches over time for the same query. This work presents a new, alternative approach to search query bias identification that includes less high-level suggestions to deepen the data basis of bias analyses. We employ recursive algorithm interrogation techniques and create suggestion trees that enable access to more subliminal search query suggestions. Based on these suggestions, we investigate topical group bias in person-related searches in the political domain.
Paper Structure (16 sections, 1 equation, 2 figures, 3 tables)

This paper contains 16 sections, 1 equation, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Auditing Search Query Bias
  4. Query Suggestion Extraction with Recursive Algorithm Interrogation
  5. Methodological Approach
  6. Dataset Creation and Preprocessing
  7. Selection of Root Terms and Suggestion Crawl
  8. Preprocessing: Cleaning data
  9. Preprocessing: Vectorization
  10. Semantic Clustering
  11. Resulting Clusters
  12. Topical Bias Analysis
  13. Findings on Bias in Political Query Suggestions
  14. Discussion
  15. Outlook
  16. ...and 1 more sections

Figures (2)

  • Figure 1: Schematic view of the methodological approach for RAI-based bias detection using suggestion trees introduced by robertson_auditing_2019.
  • Figure 2: Example for a suggestion tree based on the root term annalena baerbock (the German Federal Foreign Minister, former co-leader of the Green Party), marked in green. Suggestion terms added to a former query string are marked with +; otherwise, the substituted query string is entirely written out. In order to preserve the clarity of the figure, some labels have been omitted. The dark grey nodes show suggested terms based on the former query. Light grey nodes are suggestions based on a first letter suggestion or their subsequent suggestions. Nodes with suggestions unrelated to the initial root query are marked in red. These nodes are pruned, including their following sub-trees.