Choosing Between an LLM versus Search for Learning: A HigherEd Student Perspective

TL;DR

The study investigates whether learning with ChatGPT competes with or complements traditional search-based learning in higher education. Using a within-subjects design with 20 university students, it compares experiences across interfaces, synthesis, prompting, task goals, and integrity when learning a new topic with ChatGPT versus Google. Results indicate ChatGPT provides structured, narrative synthesis and on-demand context that supports writing and comprehension, but raises concerns about accuracy, transparency, and retention; Google offers broad, verifiable sources and incidental learning, yet can overwhelm and lacks integrated synthesis. Overall, the findings suggest these tools are complementary rather than mutually exclusive, with implications for instructional design, AI literacy, and policy-making to guide future classroom use and integrity frameworks.

Abstract

Large language models (LLMs) are rapidly changing learning processes, as they are readily available to students and quickly complete or augment several learning-related activities with non-trivial performance. Such major shifts in learning dynamic have previously occurred when search engines and Wikipedia were introduced, and they augmented or traditional information consumption sources such as libraries and books for university students. We investigate the possibility of the next shift: the use of LLMs to find and digest information in the context of learning and how they relate to existing technologies such as the search engine. We conducted a study where students were asked to learn new topics using a search engine and an LLM in a within-subjects counterbalanced design. We used that study as a contextual grounding for a post-experience follow-up interview where we elicited student reflections, preferences, pain points, and general outlook of an LLM (ChatGPT) over a search engine (Google).