A Survey on Failure Analysis and Fault Injection in AI Systems

TL;DR

This work provides the first comprehensive survey of failure analysis and fault injection across AI systems, organizing FA/FI across six layers (service, model, framework, toolkit, platform, infrastructure) and synthesizing 142 studies from 2001 to 2024. It builds a taxonomy of AI system failures, evaluates the current capabilities of FI tools, and identifies substantial gaps between simulated and real-world failures, especially regarding cross-layer dependencies and platform-specific faults. By detailing methodologies, tool ecosystems, and cross-layer gaps, the paper offers a structured framework for fault diagnosis and a roadmap for advancing fault injection techniques to bolster AI resilience. The findings have practical significance for developers, researchers, and operators seeking to improve the reliability and safety of AI-enabled systems, including large language model deployments and production AI services.

Abstract

The rapid advancement of Artificial Intelligence (AI) has led to its integration into various areas, especially with Large Language Models (LLMs) significantly enhancing capabilities in Artificial Intelligence Generated Content (AIGC). However, the complexity of AI systems has also exposed their vulnerabilities, necessitating robust methods for failure analysis (FA) and fault injection (FI) to ensure resilience and reliability. Despite the importance of these techniques, there lacks a comprehensive review of FA and FI methodologies in AI systems. This study fills this gap by presenting a detailed survey of existing FA and FI approaches across six layers of AI systems. We systematically analyze 160 papers and repositories to answer three research questions including (1) what are the prevalent failures in AI systems, (2) what types of faults can current FI tools simulate, (3) what gaps exist between the simulated faults and real-world failures. Our findings reveal a taxonomy of AI system failures, assess the capabilities of existing FI tools, and highlight discrepancies between real-world and simulated failures. Moreover, this survey contributes to the field by providing a framework for fault diagnosis, evaluating the state-of-the-art in FI, and identifying areas for improvement in FI techniques to enhance the resilience of AI systems.

Figures (6)

• Figure 1: An overall framework of AI system.
• Figure 2: Failure comparisons between AI and cloud systems.
• Figure 3: Overall life cycle of an AI system failure.
• Figure 4: Search strategy of our study.
• Figure 5: Distribution of studies across layers.