Precision Proactivity: Measuring Cognitive Load in Real-World AI-Assisted Work

TL;DR

Drawing on Cognitive Load Theory, a transcript-based framework is developed estimating intrinsic and extraneous load from computational indicators anchored in a task decomposition and knowledge graph and reveals a compensatory pathway partially offsetting but not eliminating load-related deficits.

Abstract

Systems like ChatGPT and Claude assist billions through proactive dialogue-offering unsolicited, task-relevant information. Drawing on Cognitive Load Theory, we study how cognitive load shapes performance in AI-assisted knowledge work. We recruited 34 financial professionals to complete a complex valuation task using GPT-4o and developed a transcript-based framework estimating intrinsic and extraneous load from computational indicators anchored in a task decomposition and knowledge graph. Across 1,178 participant-subtask observations, AI-generated content usage is positively associated with quality, while extraneous load shows the largest negative association-roughly three times that of intrinsic load. Mediation reveals a compensatory pathway partially offsetting but not eliminating load-related deficits. Extraneous load persists within speakers and spills asymmetrically to model responses. Model-initiated task switching is the strongest predictor of decline. Expertise moderates these dynamics: less experienced professionals face larger penalties and derive greater marginal gains from AI-generated content, yet are not those who most increase uptake under load.

Figures (2)

• Figure 1: Example turn-level annotation of a user--LLM conversation using our computational framework. For each prompt--response pair, we estimate intrinsic and extraneous cognitive load from the transcript and compute interaction-level indicators of structural misalignment (e.g., task switching, information sprawl, phase spread). These measures are derived from task-decomposition structure and semantic coherence and enable fine-grained analysis of how cognitive burden and conversational organization evolve within and across speakers.
• Figure 2: Overview of Computational Pipeline Top: Construction of the Finance Knowledge Graph (FinanceKG), including document collection, concept extraction, embedding, and graph formation. The resulting graph provides structured semantic distances and concept anchors used in extraneous load calculations (see SI § S6). Middle: Participant–subtask pipeline for computing utterance-level Intrinsic Load (IL), Extraneous Load (EL), and behavioral features. This stage integrates subtask memory vectors, FinanceKG-based concept distances, autoregressive memory blending, and coherence penalties. All load measures are standardized at the utterance level prior to panel modeling (see SI § S4). Bottom: Attribution pipeline for estimating AI-generated content usage (AIGCU) in the final valuation report. Transcript sentences are matched to report sentences under capacitated, thresholded similarity constraints. Content usage metrics are computed independently of the load measures (see SI § S5).