Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AI summaries in online search influence users' attitudes

Yiwei Xu, Saloni Dash, Sungha Kang, Wang Liao, Emma S. Spiro

TL;DR

This study experimentally tests how AI-generated summaries embedded in search results influence users' attitudes, behavioral intentions, and policy support across four health topics. By manipulating the presence, placement, and framing (benefit vs harm) of AI summaries, the researchers show that AI summaries can bias judgments toward the summarized stance, with top placement amplifying attitude shifts. Moderators including issue familiarity and general trust in AI modulate these effects, and harm-framed summaries are perceived as more useful. The findings highlight important design and regulatory considerations for AI-enabled information ecosystems and the potential for heuristic processing to shape public opinion online.

Abstract

This study examined how AI-generated summaries, which have become visually prominent in online search results, affect how users think about different issues. In a preregistered randomized controlled experiment, participants (N = 2,004) viewed mock search result pages varying in the presence (vs. absence), placement (top vs. middle), and stance (benefit-framed vs. harm-framed) of AI-generated summaries across four publicly debated topics. Compared to a no-summary control group, participants exposed to AI-generated summaries reported issue attitudes, behavioral intentions, and policy support that aligned more closely with the AI summary stance. The summaries placed at the top of the page produced stronger shifts in users' issue attitudes (but not behavioral intentions or policy support) than those placed at the middle of the page. We also observed moderating effects from issue familiarity and general trust toward AI. In addition, users perceived the AI summaries more useful when it emphasized health harms versus benefits. These findings suggest that AI-generated search summaries can significantly shape public perceptions, raising important implications for the design and regulation of AI-integrated information ecosystems.

AI summaries in online search influence users' attitudes

TL;DR

This study experimentally tests how AI-generated summaries embedded in search results influence users' attitudes, behavioral intentions, and policy support across four health topics. By manipulating the presence, placement, and framing (benefit vs harm) of AI summaries, the researchers show that AI summaries can bias judgments toward the summarized stance, with top placement amplifying attitude shifts. Moderators including issue familiarity and general trust in AI modulate these effects, and harm-framed summaries are perceived as more useful. The findings highlight important design and regulatory considerations for AI-enabled information ecosystems and the potential for heuristic processing to shape public opinion online.

Abstract

This study examined how AI-generated summaries, which have become visually prominent in online search results, affect how users think about different issues. In a preregistered randomized controlled experiment, participants (N = 2,004) viewed mock search result pages varying in the presence (vs. absence), placement (top vs. middle), and stance (benefit-framed vs. harm-framed) of AI-generated summaries across four publicly debated topics. Compared to a no-summary control group, participants exposed to AI-generated summaries reported issue attitudes, behavioral intentions, and policy support that aligned more closely with the AI summary stance. The summaries placed at the top of the page produced stronger shifts in users' issue attitudes (but not behavioral intentions or policy support) than those placed at the middle of the page. We also observed moderating effects from issue familiarity and general trust toward AI. In addition, users perceived the AI summaries more useful when it emphasized health harms versus benefits. These findings suggest that AI-generated search summaries can significantly shape public perceptions, raising important implications for the design and regulation of AI-integrated information ecosystems.

Paper Structure

This paper contains 19 sections, 6 figures, 6 tables.

Table of Contents

  1. AI Summary Presence X Issue Stance.
  2. AI Summary Placement X Issue Stance.
  3. AI Summary Presence X Familiarity.
  4. AI Summary Placement X Familiarity.
  5. AI Summary Presence X Trust.
  6. AI Summary Placement X Trust.
  7. Issue Attitudes.
  8. Behavioral Intentions.
  9. Policy Support.
  10. Issue Familiarity.
  11. AI Summary Use and Trust.
  12. General Trust in AI.
  13. Stimuli Construction and Validation
  14. Participants Information
  15. Randomization Checks
  16. ...and 4 more sections

Figures (6)

  • Figure 1: Main outcome variables by condition. (A) Issue attitudes by condition. (B) Behavioral intentions by condition. (C) Policy support by condition.
  • Figure 2: Johnson-Neyman plots with user issue familiarity as moderator. Each panel shows the estimated average outcome differences between two conditions, given a certain level of issue familiarity. Non-significantly moderated comparisons are omitted.
  • Figure 3: Johnson-Neyman plots with user general trust in AI as moderator Each panel shows the estimated average outcome differences between two conditions, given a certain level of general trust in AI. Non-significantly moderated comparisons are omitted.
  • Figure 4: Sample stimuli. Benefit-framed AI summary (raw milk) placed at the top.
  • Figure S1: Prompt for Stance Validation
  • ...and 1 more figures