Conversations over Clicks: Impact of Chatbots on Information Search in Interdisciplinary Learning

TL;DR

This study investigates how GenAI chatbots influence information search in interdisciplinary bioinformatics learning. Using autoethnography, it contrasts GenAI conversations with traditional e-tutorials to examine orienteering and information scent. Findings show GenAI supports plan-driven, focused information seeking after a task plan but imposes heavy verification loads and can misalign goals, while information scent largely hinges on prior knowledge and is weakened by generic structuring cues. The work highlights the need for cautious GenAI adoption in interdisciplinary education, emphasizing AI literacy and system-level safeguards to mitigate distraction and misinformed trust.

Abstract

This full research paper investigates the impact of generative AI (GenAI) on the learner experience, with a focus on how learners engage with and utilize the information it provides. In e-learning environments, learners often need to navigate a complex information space on their own. This challenge is further compounded in interdisciplinary fields like bioinformatics, due to the varied prior knowledge and backgrounds. In this paper, we studied how GenAI influences information search in bioinformatics research: (1) How do interactions with a GenAI chatbot influence learner orienteering behaviors?; and (2) How do learners identify information scent in GenAI chatbot responses? We adopted an autoethnographic approach to investigate these questions. GenAI was found to support orienteering once a learning plan was established, but it was counterproductive prior to that. Moreover, traditionally value-rich information sources such as bullet points and related terms proved less effective when applied to GenAI responses. Information scents were primarily recognized through the presence or absence of prior knowledge of the domain. These findings suggest that GenAI should be adopted into e-learning environments with caution, particularly in interdisciplinary learning contexts.

Figures (3)

• Figure 1: Comparison between a traditional e-tutorial (A) and a GenAI-based conversational e-tutorial (B)
• Figure 2: Autoethnographic study protocol overview. A task scenario is provided to the first author (A). They search for information necessary to complete the task, primarily by interacting with the GenAI system (B). The interaction concludes once the task is completed (C).
• Figure 3: Quantitative summary of user queries and GenAI responses across six chat transcripts (exchange ranges: 1–14, 15–30, 31–48, 49–63, 64–75, and 76–92). The red triangle marks the point between exchanges 51 and 52, where a task completion plan was established.