Healthy Distrust in AI systems

TL;DR

The paper argues that the dominant focus on trustworthy AI overlooks justified, context-dependent distrust—termed healthy distrust—that respects human autonomy within socio-technical power structures. It advances an interdisciplinary exploration across history, philosophy, psychology, and sociology to define healthy distrust, distinguish it from mere skepticism or reliance, and analyze its normative and practical implications. By integrating concepts of risk, danger, Cartesian doubt, and epistemic vigilance, it outlines conditions under which distrust can be constructive and necessary for meaningful human oversight and informed consent. The work highlights limitations, such as unequal access to resources for distrust and potential under-utilization of AI, and calls for normative frameworks and policy support to foster healthy distrust in governance, literacy, and practice.

Abstract

Under the slogan of trustworthy AI, much of contemporary AI research is focused on designing AI systems and usage practices that inspire human trust and, thus, enhance adoption of AI systems. However, a person affected by an AI system may not be convinced by AI system design alone -- neither should they, if the AI system is embedded in a social context that gives good reason to believe that it is used in tension with a person's interest. In such cases, distrust in the system may be justified and necessary to build meaningful trust in the first place. We propose the term "healthy distrust" to describe such a justified, careful stance towards certain AI usage practices. We investigate prior notions of trust and distrust in computer science, sociology, history, psychology, and philosophy, outline a remaining gap that healthy distrust might fill and conceptualize healthy distrust as a crucial part for AI usage that respects human autonomy.