Intelligent Interaction Strategies for Context-Aware Cognitive Augmentation
Xiangrong, Zhu, Yuan Xu, Tianjian Liu, Jingwei Sun, Yu Zhang, Xin Tong
TL;DR
Context-aware cognitive augmentation with LLMs is advanced to adapt to users' cognitive states and task contexts, addressing cognitive overload. The paper reports a think-aloud study in an exhibition setting to observe interactions with multi-modal information and to identify cognitive challenges in structuring, retrieving, and applying knowledge. An AI augmentation framework is proposed, featuring real-time contextual awareness, personalized reasoning assistance, and socially adaptive interactions, with a design for transitioning between real-time support and post-experience knowledge organization. Collectively, the work contributes to designing more effective human-centered AI systems by outlining architectural and interaction strategies for seamless, context-sensitive cognitive support.
Abstract
Human cognition is constrained by processing limitations, leading to cognitive overload and inefficiencies in knowledge synthesis and decision-making. Large Language Models (LLMs) present an opportunity for cognitive augmentation, but their current reactive nature limits their real-world applicability. This position paper explores the potential of context-aware cognitive augmentation, where LLMs dynamically adapt to users' cognitive states and task environments to provide appropriate support. Through a think-aloud study in an exhibition setting, we examine how individuals interact with multi-modal information and identify key cognitive challenges in structuring, retrieving, and applying knowledge. Our findings highlight the need for AI-driven cognitive support systems that integrate real-time contextual awareness, personalized reasoning assistance, and socially adaptive interactions. We propose a framework for AI augmentation that seamlessly transitions between real-time cognitive support and post-experience knowledge organization, contributing to the design of more effective human-centered AI systems.