Trapped by Expectations: Functional Fixedness in LLM-Enabled Chat Search
Jiqun Liu, Jamshed Karimnazarov, Ryen W. White
TL;DR
The paper investigates functional fixedness in LLM-enabled chat search, examining how users' pre-chat expectations shaped by prior experiences constrain exploration. It uses a controlled crowdsourcing study with 450 participants across six decision tasks to observe interactions and adaptation when expectations are unmet. Key contributions include a typology of user intents for chat search and empirical evidence on how prior system experience affects prompting behavior and task outcomes, with implications for designing expectation-aware, exploratory interfaces. The work shows that mitigating functional fixedness can promote broader, creative use of LLMs in decision-making tasks.
Abstract
Functional fixedness, a cognitive bias that restricts users' interactions with a new system or tool to expected or familiar ways, limits the full potential of Large Language Model (LLM)-enabled chat search, especially in complex and exploratory tasks. To investigate its impact, we conducted a crowdsourcing study with 450 participants, each completing one of six decision-making tasks spanning public safety, diet and health management, sustainability, and AI ethics. Participants engaged in a multi-prompt conversation with ChatGPT to address the task, allowing us to compare pre-chat intent-based expectations with observed interactions. We found that: 1) Several aspects of pre-chat expectations are closely associated with users' prior experiences with ChatGPT, search engines, and virtual assistants; 2) Prior system experience shapes language use and prompting behavior. Frequent ChatGPT users reduced deictic terms and hedge words and frequently adjusted prompts. Users with rich search experience maintained structured, less-conversational queries with minimal modifications. Users of virtual assistants favored directive, command-like prompts, reinforcing functional fixedness; 3) When the system failed to meet expectations, participants generated more detailed prompts with increased linguistic diversity, reflecting adaptive shifts. These findings suggest that while preconceived expectations constrain early interactions, unmet expectations can motivate behavioral adaptation. With appropriate system support, this may promote broader exploration of LLM capabilities. This work also introduces a typology for user intents in chat search and highlights the importance of mitigating functional fixedness to support more creative and analytical use of LLMs.