Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Understanding and Avoiding AI Failures: A Practical Guide

Heather M. Williams, Roman V. Yampolskiy

TL;DR

This work creates a framework for understanding the risks associated with AI applications, and uses AI safety principles to quantify the unique risks of increased intelligence and human-like qualities in AI.

Abstract

As AI technologies increase in capability and ubiquity, AI accidents are becoming more common. Based on normal accident theory, high reliability theory, and open systems theory, we create a framework for understanding the risks associated with AI applications. In addition, we also use AI safety principles to quantify the unique risks of increased intelligence and human-like qualities in AI. Together, these two fields give a more complete picture of the risks of contemporary AI. By focusing on system properties near accidents instead of seeking a root cause of accidents, we identify where attention should be paid to safety for current generation AI systems.

Understanding and Avoiding AI Failures: A Practical Guide

TL;DR

This work creates a framework for understanding the risks associated with AI applications, and uses AI safety principles to quantify the unique risks of increased intelligence and human-like qualities in AI.

Abstract

As AI technologies increase in capability and ubiquity, AI accidents are becoming more common. Based on normal accident theory, high reliability theory, and open systems theory, we create a framework for understanding the risks associated with AI applications. In addition, we also use AI safety principles to quantify the unique risks of increased intelligence and human-like qualities in AI. Together, these two fields give a more complete picture of the risks of contemporary AI. By focusing on system properties near accidents instead of seeking a root cause of accidents, we identify where attention should be paid to safety for current generation AI systems.

Paper Structure

This paper contains 49 sections, 1 figure, 14 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Early History
  4. Normal Accident Theory
  5. High Reliability Theory
  6. NAT-HRT Reconciliation
  7. Lethal Autonomous Weaponry
  8. Robustness Issues for Learned Agents
  9. Examples of AI Failures
  10. Societal Impact of AI
  11. Engineered Interpretability Measures
  12. Faulty Post-hoc Explanations in Humans
  13. The AI Accident Network
  14. AI Safety
  15. Avoiding Negative Side Effects
  16. ...and 34 more sections

Figures (1)

  • Figure 1: Plotting energy level against knowledge gap to create 4 quadrants. Systems in quadrant 3 have the lowest damage potential, and damage is limited to 1st parties. Systems in quadrant 4 have moderate damage potential up to 4th party victims, systems in quadrant 1 have high damage potential up to third party victims, and systems in quadrant 1 have catastrophic risk potentials to 4th party victims. Based on Shrivastava et al. shrivastava2009normal.