Dissecting users' needs for search result explanations

TL;DR

Dissecting users' needs for search result explanations investigates whether and when explanations improve search experiences for non-technical users. The study uses a two-phase, mixed-methods approach (online surveys and semi-structured interviews) to identify situations where explanations are desired, the benefits users expect, and the characteristics they value. Key findings show explanations are not universally needed; they are most helpful for complex, high-stakes, or real-world product/service tasks, and users prefer concise, actionable explanations with credibility signals and user agency (e.g., ability to contest results). The work offers practical design recommendations for search engines to enhance interpretable transparency, including content categorization, process guidance, viewpoint indication, and improved discoverability of explanation features, with implications for trust and search efficiency.

Abstract

There is a growing demand for transparency in search engines to understand how search results are curated and to enhance users' trust. Prior research has introduced search result explanations with a focus on how to explain, assuming explanations are beneficial. Our study takes a step back to examine if search explanations are needed and when they are likely to provide benefits. Additionally, we summarize key characteristics of helpful explanations and share users' perspectives on explanation features provided by Google and Bing. Interviews with non-technical individuals reveal that users do not always seek or understand search explanations and mostly desire them for complex and critical tasks. They find Google's search explanations too obvious but appreciate the ability to contest search results. Based on our findings, we offer design recommendations for search engines and explanations to help users better evaluate search results and enhance their search experience.

Figures (3)

• Figure 1: (a) Figure showing highlighted keywords in the snippet of text accompanying the search result, offering insights into the alignment of the search result with the user's search query, (b) Figure showing search explanations provided as an example during the interview. This example illustrates an overall explanation describing how the search engine works and, in addition, explains how relevance influences the ranking of search results.
• Figure 2: Figures \ref{['g1']} and \ref{['g2']} demonstrate the search result explanations and additional context presented by the Google search engine alongside each search result. These details can be accessed by clicking on the three vertical dots next to each search result as shown in Figure \ref{['g_dots']}. The provided data includes various features such as the option to remove a result and provide feedback on the search result page, a Wikipedia link for accessing supplementary information about the source, a link to privacy/personalization settings, an explanation for the appearance of the specific search result, and a link that directs users to a page offering a broad overview of how search results are curated by search engines. On the other hand, Figure \ref{['bing']} illustrates the search context provided by Bing. This can be accessed by clicking on the light bulb present along each search result as shown in Figure \ref{['bing_bulb']}. It encompasses details about the source (if available on Wikipedia/Encyclopedia), a preview of relevant topics on the webpage, and additional recommendations associated with the search result.
• Figure 3: Google's topic filters feature introduced in 2023 Googlero39:online