Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Establishing and Evaluating Trustworthy AI: Overview and Research Challenges

Dominik Kowald, Sebastian Scher, Viktoria Pammer-Schindler, Peter Müllner, Kerstin Waxnegger, Lea Demelius, Angela Fessl, Maximilian Toller, Inti Gabriel Mendoza Estrada, Ilija Simic, Vedran Sabol, Andreas Truegler, Eduardo Veas, Roman Kern, Tomislav Nad, Simone Kopeinik

TL;DR

This paper synthesizes and consolidates a wide-ranging and active discussion currently taking place in various academic sub-communities and public forums about aspects that AI systems must fulfill to be considered trustworthy.

Abstract

Artificial intelligence (AI) technologies (re-)shape modern life, driving innovation in a wide range of sectors. However, some AI systems have yielded unexpected or undesirable outcomes or have been used in questionable manners. As a result, there has been a surge in public and academic discussions about aspects that AI systems must fulfill to be considered trustworthy. In this paper, we synthesize existing conceptualizations of trustworthy AI along six requirements: 1) human agency and oversight, 2) fairness and non-discrimination, 3) transparency and explainability, 4) robustness and accuracy, 5) privacy and security, and 6) accountability. For each one, we provide a definition, describe how it can be established and evaluated, and discuss requirement-specific research challenges. Finally, we conclude this analysis by identifying overarching research challenges across the requirements with respect to 1) interdisciplinary research, 2) conceptual clarity, 3) context-dependency, 4) dynamics in evolving systems, and 5) investigations in real-world contexts. Thus, this paper synthesizes and consolidates a wide-ranging and active discussion currently taking place in various academic sub-communities and public forums. It aims to serve as a reference for a broad audience and as a basis for future research directions.

Establishing and Evaluating Trustworthy AI: Overview and Research Challenges

TL;DR

This paper synthesizes and consolidates a wide-ranging and active discussion currently taking place in various academic sub-communities and public forums about aspects that AI systems must fulfill to be considered trustworthy.

Abstract

Artificial intelligence (AI) technologies (re-)shape modern life, driving innovation in a wide range of sectors. However, some AI systems have yielded unexpected or undesirable outcomes or have been used in questionable manners. As a result, there has been a surge in public and academic discussions about aspects that AI systems must fulfill to be considered trustworthy. In this paper, we synthesize existing conceptualizations of trustworthy AI along six requirements: 1) human agency and oversight, 2) fairness and non-discrimination, 3) transparency and explainability, 4) robustness and accuracy, 5) privacy and security, and 6) accountability. For each one, we provide a definition, describe how it can be established and evaluated, and discuss requirement-specific research challenges. Finally, we conclude this analysis by identifying overarching research challenges across the requirements with respect to 1) interdisciplinary research, 2) conceptual clarity, 3) context-dependency, 4) dynamics in evolving systems, and 5) investigations in real-world contexts. Thus, this paper synthesizes and consolidates a wide-ranging and active discussion currently taking place in various academic sub-communities and public forums. It aims to serve as a reference for a broad audience and as a basis for future research directions.

Paper Structure

This paper contains 37 sections, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Definitions and Preliminaries of Trustworthy AI
  4. Requirements of Trustworthy AI
  5. Methodology
  6. Overview and Discussion of Trustworthy AI Requirements
  7. Requirement 1: Human Agency and Oversight
  8. Definition of Human Agency and Oversight
  9. Methods to Establish Human Agency and Oversight
  10. Methods to Evaluate Human Agency and Oversight
  11. Open Issues and Research Challenges
  12. Requirement 2: Fairness and Non-Discrimination
  13. Definition of Fairness and Non-Discrimination
  14. Methods to Establish Fairness and Non-Discrimination
  15. Methods to Evaluate Fairness and Non-Discrimination
  16. ...and 22 more sections

Figures (4)

  • Figure 1: An illustration of the six requirements of trustworthy AI investigated in this paper.
  • Figure 2: The AI-lifecycle. The trustworthiness of AI can be conflicted in all phases - the design phase, the development phase, and the deployment phase.
  • Figure 3: The number of publications per requirement included in this paper across publication years. We investigate 183 publications: 20 publications for human agency and oversight, 35 publications for fairness and non-discrimination, 47 publications for Transparency and explainability, 21 publications for robustness and accuracy, 37 publications for privacy and security, and 23 publications for accountability.
  • Figure 4: Overarching research challenges identified in this paper in relation to the AI-lifecycle phases.