A Design Framework for operationalizing Trustworthy Artificial Intelligence in Healthcare: Requirements, Tradeoffs and Challenges for its Clinical Adoption
Pedro A. Moreno-Sánchez, Javier Del Ser, Mark van Gils, Jussi Hernesniemi
TL;DR
The paper tackles the challenge of deploying trustworthy AI in healthcare by proposing a design-by-design framework that translates Trustworthy AI principles into concrete requirements across medical processes, data types, and stakeholder roles, with a concrete cardiovascular disease use case. It integrates human agency, robustness, privacy, transparency, fairness, sustainability, and accountability, mapping them to actionable design criteria, MoSCoW prioritization, and stakeholder matrices. The framework is complemented by discussions of tradeoffs, regulatory alignment (GDPR, AI Act, ISO/NIST standards), and practical challenges, aiming to guide AI developers and healthcare stakeholders toward safer, more interpretable, and governance-compliant AI adoption. The work emphasizes that successful clinical integration hinges on early stakeholder involvement, rigorous validation, and adaptable evaluation approaches to certify trustworthiness throughout the AI lifecycle, particularly in high-stakes domains like cardiology.
Abstract
Artificial Intelligence (AI) holds great promise for transforming healthcare, particularly in disease diagnosis, prognosis, and patient care. The increasing availability of digital medical data, such as images, omics, biosignals, and electronic health records, combined with advances in computing, has enabled AI models to approach expert-level performance. However, widespread clinical adoption remains limited, primarily due to challenges beyond technical performance, including ethical concerns, regulatory barriers, and lack of trust. To address these issues, AI systems must align with the principles of Trustworthy AI (TAI), which emphasize human agency and oversight, algorithmic robustness, privacy and data governance, transparency, bias and discrimination avoidance, and accountability. Yet, the complexity of healthcare processes (e.g., screening, diagnosis, prognosis, and treatment) and the diversity of stakeholders (clinicians, patients, providers, regulators) complicate the integration of TAI principles. To bridge the gap between TAI theory and practical implementation, this paper proposes a design framework to support developers in embedding TAI principles into medical AI systems. Thus, for each stakeholder identified across various healthcare processes, we propose a disease-agnostic collection of requirements that medical AI systems should incorporate to adhere to the principles of TAI. Additionally, we examine the challenges and tradeoffs that may arise when applying these principles in practice. To ground the discussion, we focus on cardiovascular diseases, a field marked by both high prevalence and active AI innovation, and demonstrate how TAI principles have been applied and where key obstacles persist.