Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Position: Human-Centric AI Requires a Minimum Viable Level of Human Understanding

Fangzhou Lin, Qianwen Ge, Lingyu Xu, Peiran Li, Xiangbo Gao, Shuo Xing, Kazunori Yamada, Ziming Zhang, Haichong Zhang, Zhengzhong Tu

TL;DR

The paper addresses the risk that AI-enabled reasoning gradually erodes human task-relevant understanding, producing a Capability-Comprehension Gap where oversight becomes hollow. It defines the Cognitive Integrity Threshold (CIT) as the minimum viable level of understanding to sustain verification, reconstruction, and boundary awareness under AI assistance, and it articulates three functional capacities (V, R, B) that constitute CIT. The authors propose a reusable operational template, independence requirements, and domain instantiations to operationalize CIT, along with Comprehension-Preserving Interaction (CPI), broader evaluation, and governance mechanisms to maintain cognitive recoverability. The work aims to shift design and policy from merely improving AI performance to preserving human accountability and controllability in responsibility-critical settings such as healthcare, law, and public infrastructure.

Abstract

AI systems increasingly produce fluent, correct, end-to-end outcomes. Over time, this erodes users' ability to explain, verify, or intervene. We define this divergence as the Capability-Comprehension Gap: a decoupling where assisted performance improves while users' internal models deteriorate. This paper argues that prevailing approaches to transparency, user control, literacy, and governance do not define the foundational understanding humans must retain for oversight under sustained AI delegation. To formalize this, we define the Cognitive Integrity Threshold (CIT) as the minimum comprehension required to preserve oversight, autonomy, and accountable participation under AI assistance. CIT does not require full reasoning reconstruction, nor does it constrain automation. It identifies the threshold beyond which oversight becomes procedural and contestability fails. We operatinalize CIT through three functional dimensions: (i) verification capacity, (ii) comprehension-preserving interaction, and (iii) institutional scaffolds for governance. This motivates a design and governance agenda that aligns human-AI interaction with cognitive sustainability in responsibility-critical settings.

Position: Human-Centric AI Requires a Minimum Viable Level of Human Understanding

TL;DR

The paper addresses the risk that AI-enabled reasoning gradually erodes human task-relevant understanding, producing a Capability-Comprehension Gap where oversight becomes hollow. It defines the Cognitive Integrity Threshold (CIT) as the minimum viable level of understanding to sustain verification, reconstruction, and boundary awareness under AI assistance, and it articulates three functional capacities (V, R, B) that constitute CIT. The authors propose a reusable operational template, independence requirements, and domain instantiations to operationalize CIT, along with Comprehension-Preserving Interaction (CPI), broader evaluation, and governance mechanisms to maintain cognitive recoverability. The work aims to shift design and policy from merely improving AI performance to preserving human accountability and controllability in responsibility-critical settings such as healthcare, law, and public infrastructure.

Abstract

AI systems increasingly produce fluent, correct, end-to-end outcomes. Over time, this erodes users' ability to explain, verify, or intervene. We define this divergence as the Capability-Comprehension Gap: a decoupling where assisted performance improves while users' internal models deteriorate. This paper argues that prevailing approaches to transparency, user control, literacy, and governance do not define the foundational understanding humans must retain for oversight under sustained AI delegation. To formalize this, we define the Cognitive Integrity Threshold (CIT) as the minimum comprehension required to preserve oversight, autonomy, and accountable participation under AI assistance. CIT does not require full reasoning reconstruction, nor does it constrain automation. It identifies the threshold beyond which oversight becomes procedural and contestability fails. We operatinalize CIT through three functional dimensions: (i) verification capacity, (ii) comprehension-preserving interaction, and (iii) institutional scaffolds for governance. This motivates a design and governance agenda that aligns human-AI interaction with cognitive sustainability in responsibility-critical settings.
Paper Structure (38 sections, 1 figure, 2 tables)

This paper contains 38 sections, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Empirical Foundation and Problem Background
  3. Mechanism: How reasoning automation erodes comprehension
  4. Thresholded failure: Why oversight collapses abruptly
  5. Limitations of existing paradigms
  6. Distinguishing CIT from Automation Bias, Complacency, and Deskilling
  7. The Cognitive Integrity Threshold (CIT)
  8. Definition: CIT as minimum viable oversight capacity
  9. Three capacities that constitute CIT
  10. CIT vs. adjacent ideas
  11. CIT as a deployment contract
  12. CIT Operationalization in Critical Domains
  13. A reusable operational template
  14. Why "independent" matters
  15. Representative instantiations
  16. ...and 23 more sections

Figures (1)

  • Figure 1: The CIT Model. The figure illustrates the divergence between AI-augmented performance and retained human task-understanding as a function of increasing AI delegation. Once human comprehension drops below the CIT, oversight becomes structurally hollow—entering a regime of Empty Oversight. The black dot marks the Critical Decoupling Point, where the operator can no longer reconstruct or recover system intent in the face of anomalies, despite nominal control.