From Consumption to Collaboration: Measuring Interaction Patterns to Augment Human Cognition in Open-Ended Tasks

TL;DR

The paper addresses how Generative AI and LLMs influence human cognition in open-ended tasks, where ground truth is absent and iterative collaboration with AI can be either beneficial or harmful. It introduces a two-dimensional framework combining cognitive activity mode (exploration vs exploitation) with cognitive engagement mode (constructive vs detrimental) and pairs it with a measurement approach—dialogue segmentation, mode-transition detection, and engagement indicators—to quantify human–LLM interaction quality. The core contributions are the four interaction patterns (Constructive Exploration, Constructive Exploitation, Detrimental Exploration, Detrimental Exploitation), a practical methodology for real-time analysis (potentially automated via LLMs), and design implications for creating AI systems that augment rather than erode cognition. The work advances theoretical understanding and offers practical guidance for developing interfaces and interventions that promote reflective, context-rich collaboration in open-ended cognitive tasks, with empirical validation identified as a key future step.

Abstract

The rise of Generative AI, and Large Language Models (LLMs) in particular, is fundamentally changing cognitive processes in knowledge work, raising critical questions about their impact on human reasoning and problem-solving capabilities. As these AI systems become increasingly integrated into workflows, they offer unprecedented opportunities for augmenting human thinking while simultaneously risking cognitive erosion through passive consumption of generated answers. This tension is particularly pronounced in open-ended tasks, where effective solutions require deep contextualization and integration of domain knowledge. Unlike structured tasks with established metrics, measuring the quality of human-LLM interaction in such open-ended tasks poses significant challenges due to the absence of ground truth and the iterative nature of solution development. To address this, we present a framework that analyzes interaction patterns along two dimensions: cognitive activity mode (exploration vs. exploitation) and cognitive engagement mode (constructive vs. detrimental). This framework provides systematic measurements to evaluate when LLMs are effective tools for thought rather than substitutes for human cognition, advancing theoretical understanding and practical guidance for developing AI systems that protect and augment human cognitive capabilities.

Figures (2)

• Figure 1: We distinguish between four types of human-LLM interaction across the two dimensions of cognitive activity mode and cognitive engagement mode.
• Figure 2: An exemplary chat history, where an LLM coded the user interaction along our two dimensions: cognitive activity and cognitive engagement mode.