Whether to trust: the ML leap of faith

TL;DR

The paper addresses the challenge that attitudinal trust poorly predicts real-world adoption of trustworthy ML. It introduces Leap of Faith (LoF) and the LoF matrix to quantify intrinsic trust by comparing a user’s mental model with an expert-validated rules-based reference in a neuro-symbolic, parallel AI architecture. Trust is measured with three metrics—DIRTI, DAFTI, and DOTI—that assess demonstrated and deserved trust based on actions and outcomes, not self-reported intent. In a 3-month sleep-improvement pilot, the framework enables data validation, user-driven objective functions, and explicit visualization of agreement or disagreement between ML and rules-based recommendations, yielding actionable insights for managing trust and improving high-stakes adoption.

Abstract

Human trust is a prerequisite to trustworthy AI adoption, yet trust remains poorly understood. Trust is often described as an attitude, but attitudes cannot be reliably measured or managed. Additionally, humans frequently conflate trust in an AI system, its machine learning (ML) technology, and its other component parts. Without fully understanding the 'leap of faith' involved in trusting ML, users cannot develop intrinsic trust in these systems. A common approach to building trust is to explain a ML model's reasoning process. However, such explanations often fail to resonate with non-experts due to the inherent complexity of ML systems and explanations are disconnected from users' own (unarticulated) mental models. This work puts forward an innovative way of directly building intrinsic trust in ML, by discerning and measuring the Leap of Faith (LoF) taken when a user decides to rely on ML. The LoF matrix captures the alignment between an ML model and a human expert's mental model. This match is rigorously and practically identified by feeding the user's data and objective function into both an ML agent and an expert-validated rules-based agent: a verified point of reference that can be tested a priori against a user's own mental model. This represents a new class of neuro-symbolic architecture. The LoF matrix reveals to the user the distance that constitutes the leap of faith between the rules-based and ML agents. For the first time, we propose trust metrics that evaluate whether users demonstrate trust through their actions rather than self-reported intent and whether such trust is deserved based on outcomes. The significance of the contribution is that it enables empirical assessment and management of ML trust drivers, to support trustworthy ML adoption. The approach is illustrated through a long-term high-stakes field study: a 3-month pilot of a multi-agent sleep-improvement system.

Figures (11)

• Figure 1: Trust and trustworthiness definitions. All human-centred concepts are shown in yellow; technology ones in blue; combined in green. Icons by Angriawan Ditya Zulkamain, Arif Hariyanto, Karen Tyler and PEBIAN from thenounproject.com.
• Figure 2: Socio-technical sleep-improvement system with parallel neuro-symbolic AI.
• Figure 3: Caffeine data validation example: upfront data-quality trial chart on the left; baseline chart on the right. The rules embedded in the rules-based model were used to colour-code the latter – in this example, blue for <230mg; red for >400mg; yellow for in-between.
• Figure 4: Unfamiliar-data example recorded while the participant was asleep: bedside temperature on the left; bedside illumination on the right. The nightly average is shown as a horizontal mid-blue line.
• Figure 5: lofm matrix for a pilot participant.