Limits of trust in medical AI

TL;DR

The paper addresses the tension between promising medical AI capabilities and the potential erosion of doctor–patient trust. It uses a philosophical analysis of trust, reliability, agency, and epistemic authority to argue that AI cannot be genuinely trusted or be the object of trust in the interpersonal sense. It contends that AI-driven displacement of clinicians could weaken the doctor–patient relationship unless trust remains rooted in human agents. Practically, it advocates reframing AI goals toward reliability and ensuring deployment preserves the interpersonal trust essential to medical care.

Abstract

Artificial intelligence (AI) is expected to revolutionize the practice of medicine. Recent advancements in the field of deep learning have demonstrated success in a variety of clinical tasks: detecting diabetic retinopathy from images, predicting hospital readmissions, aiding in the discovery of new drugs, etc. AI's progress in medicine, however, has led to concerns regarding the potential effects of this technology upon relationships of trust in clinical practice. In this paper, I will argue that there is merit to these concerns, since AI systems can be relied upon, and are capable of reliability, but cannot be trusted, and are not capable of trustworthiness. Insofar as patients are required to rely upon AI systems for their medical decision-making, there is potential for this to produce a deficit of trust in relationships in clinical practice.