Designing AI Systems that Augment Human Performed vs. Demonstrated Critical Thinking

TL;DR

This paper addresses how GenAI affects human critical thinking by distinguishing performed from demonstrated critical thinking. It reviews existing definitions and evaluation methods, then argues that most studies measure demonstrated thinking and neglect performed thinking. Drawing on Engelbart's intellect augmentation framework and related work, it outlines design paths to augment either performed or demonstrated thinking and discusses practical evaluation strategies. The proposed distinction reframes AI-system design for cognitive augmentation and suggests methodological shifts toward longitudinal, independent-thinking assessment to preserve human autonomy.

Abstract

The recent rapid advancement of LLM-based AI systems has accelerated our search and production of information. While the advantages brought by these systems seemingly improve the performance or efficiency of human activities, they do not necessarily enhance human capabilities. Recent research has started to examine the impact of generative AI on individuals' cognitive abilities, especially critical thinking. Based on definitions of critical thinking across psychology and education, this position paper proposes the distinction between demonstrated and performed critical thinking in the era of generative AI and discusses the implication of this distinction in research and development of AI systems that aim to augment human critical thinking.