Understanding AI Trustworthiness: A Scoping Review of AIES & FAccT Articles

TL;DR

This scoping review critiques a predominantly techno-centric literature on AI trustworthiness by synthesizing AIES and FAccT scholarship. It combines qualitative thematic analysis with corpus methods to map definitions, drivers, measurement practices, and value embeddings, revealing a persistent sociotechnical gap. The authors highlight a need for longitudinal, context-aware, and participatory approaches that account for power dynamics, governance, and societal impact, while offering actionable recommendations across high-stakes domains. By articulating a holistic research agenda, the paper aims to reduce ethics washing and promote AI development that aligns with broad societal values. Overall, the work advances a more robust, interdisciplinary understanding of trustworthy AI with practical guidance for researchers, practitioners, and policymakers.

Abstract

Background: Trustworthy AI serves as a foundational pillar for two major AI ethics conferences: AIES and FAccT. However, current research often adopts techno-centric approaches, focusing primarily on technical attributes such as reliability, robustness, and fairness, while overlooking the sociotechnical dimensions critical to understanding AI trustworthiness in real-world contexts. Objectives: This scoping review aims to examine how the AIES and FAccT communities conceptualize, measure, and validate AI trustworthiness, identifying major gaps and opportunities for advancing a holistic understanding of trustworthy AI systems. Methods: We conduct a scoping review of AIES and FAccT conference proceedings to date, systematically analyzing how trustworthiness is defined, operationalized, and applied across different research domains. Our analysis focuses on conceptualization approaches, measurement methods, verification and validation techniques, application areas, and underlying values. Results: While significant progress has been made in defining technical attributes such as transparency, accountability, and robustness, our findings reveal critical gaps. Current research often predominantly emphasizes technical precision at the expense of social and ethical considerations. The sociotechnical nature of AI systems remains less explored and trustworthiness emerges as a contested concept shaped by those with the power to define it. Conclusions: An interdisciplinary approach combining technical rigor with social, cultural, and institutional considerations is essential for advancing trustworthy AI. We propose actionable measures for the AI ethics community to adopt holistic frameworks that genuinely address the complex interplay between AI systems and society, ultimately promoting responsible technological development that benefits all stakeholders.

Figures (5)

• Figure 1: Flowchart of the articles reviewing process following the PRISMA protocol page2021prisma.
• Figure 2: Heatmap of the normalized frequencies of trustworthiness dimensions in each paper of the final corpus ($N=43$).
• Figure 3: Visualizations of topics generated using BERTopic applied to the initial keyword-based selection of articles ($N=235$). First, the heatmap (left) shows normalized topic distributions, grouped by year. Values refer to the proportion of documents associated with the topic. Second, the intertopic distance plot (right) shows the relative positions of topics in a 2D space.
• Figure 4: Understanding of AI Trustworthiness: Key themes in definitions and conceptualizations.
• Figure 5: The drivers of AI trustworthiness of the primary studies covered by the review.