"Can You Tell Me?": Designing Copilots to Support Human Judgement in Online Information Seeking

TL;DR

The paper addresses the risk that Generative AI can encourage overreliance during online information seeking and proposes a literacy-focused, Socratic copilot that scaffolds evaluation rather than providing direct answers. Using a preregistered randomized controlled trial with three interface conditions (10-blue-links, ai-overview, copilot) and N=261, the study examines whether the copilot enhances information evaluation, metacognition, and search strategies in health-related queries. Results show that while users deeply engaged with the copilot and demonstrated metacognitive reflection, the copilot did not significantly improve answer correctness or search engagement relative to baseline; an inflated, biased AI overview and time spent chatting likely contributed to the null or adverse effects. The findings reveal both the potential of pedagogical copilots to foster digital-literacy skills and the practical challenges of balancing instructional friction with efficiency, informing design pathways for adaptive, transparent, and personalized evaluation-support tools.

Abstract

Generative AI (GenAI) tools are transforming information seeking, but their fluent, authoritative responses risk overreliance and discourage independent verification and reasoning. Rather than replacing the cognitive work of users, GenAI systems should be designed to support and scaffold it. Therefore, this paper introduces an LLM-based conversational copilot designed to scaffold information evaluation rather than provide answers and foster digital literacy skills. In a pre-registered, randomised controlled trial (N=261) examining three interface conditions including a chat-based copilot, our mixed-methods analysis reveals that users engaged deeply with the copilot, demonstrating metacognitive reflection. However, the copilot did not significantly improve answer correctness or search engagement, largely due to a "time-on-chat vs. exploration" trade-off and users' bias toward positive information. Qualitative findings reveal tension between the copilot's Socratic approach and users' desire for efficiency. These results highlight both the promise and pitfalls of pedagogical copilots, and we outline design pathways to reconcile literacy goals with efficiency demands.

Figures (1)

• Figure 1: Example SERP with search results on the left and the proposed interactive copilot on the right. A) Depicts the generated AI Overview for the query "Does traction help low back pain?", B) Depicts 10 search results, C) Depicts the copilot