Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SETA: Statistical Fault Attribution for Compound AI Systems

Sayak Chowdhury, Meenakshi D'Souza

TL;DR

SETA addresses the difficulty of diagnosing failures in compound AI systems by combining per-component metamorphic relations with state-based execution trace analysis. It builds a system-wide score $S(x,\tilde{x})=\prod_i S_i(x,\tilde{x})$ and module-level failure contributions $FC_i$, producing normalized attributions $\alpha_i$ to identify root-cause modules under perturbations. The framework is architecture- and modality-agnostic, demonstrated on a railway vision pipeline, an ensemble CNN, and OCR, showing how perturbations propagate and which modules contribute most to failures. This approach provides a practical, explainable pathway for robust, safer multi-model AI deployments and lays groundwork for causal fault localization in complex AI systems.

Abstract

Modern AI systems increasingly comprise multiple interconnected neural networks to tackle complex inference tasks. Testing such systems for robustness and safety entails significant challenges. Current state-of-the-art robustness testing techniques, whether black-box or white-box, have been proposed and implemented for single-network models and do not scale well to multi-network pipelines. We propose a modular robustness testing framework that applies a given set of perturbations to test data. Our testing framework supports (1) a component-wise system analysis to isolate errors and (2) reasoning about error propagation across the neural network modules. The testing framework is architecture and modality agnostic and can be applied across domains. We apply the framework to a real-world autonomous rail inspection system composed of multiple deep networks and successfully demonstrate how our approach enables fine-grained robustness analysis beyond conventional end-to-end metrics.

SETA: Statistical Fault Attribution for Compound AI Systems

TL;DR

SETA addresses the difficulty of diagnosing failures in compound AI systems by combining per-component metamorphic relations with state-based execution trace analysis. It builds a system-wide score and module-level failure contributions , producing normalized attributions to identify root-cause modules under perturbations. The framework is architecture- and modality-agnostic, demonstrated on a railway vision pipeline, an ensemble CNN, and OCR, showing how perturbations propagate and which modules contribute most to failures. This approach provides a practical, explainable pathway for robust, safer multi-model AI deployments and lays groundwork for causal fault localization in complex AI systems.

Abstract

Modern AI systems increasingly comprise multiple interconnected neural networks to tackle complex inference tasks. Testing such systems for robustness and safety entails significant challenges. Current state-of-the-art robustness testing techniques, whether black-box or white-box, have been proposed and implemented for single-network models and do not scale well to multi-network pipelines. We propose a modular robustness testing framework that applies a given set of perturbations to test data. Our testing framework supports (1) a component-wise system analysis to isolate errors and (2) reasoning about error propagation across the neural network modules. The testing framework is architecture and modality agnostic and can be applied across domains. We apply the framework to a real-world autonomous rail inspection system composed of multiple deep networks and successfully demonstrate how our approach enables fine-grained robustness analysis beyond conventional end-to-end metrics.
Paper Structure (20 sections, 6 equations, 4 figures, 3 tables, 1 algorithm)

This paper contains 20 sections, 6 equations, 4 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Metamorphic Testing for pseudo-Oracles
  4. Library of Metamorphic Relations
  5. Generic Metamorphic Relations.
  6. Composite Metamorphic Relations.
  7. System-wide Composite Score.
  8. Statistical Fault Attribution
  9. State-based Execution Trace Analysis
  10. Experimental Setup
  11. Vision System
  12. Test Dataset Generation
  13. Metamorphic Relations for Vision System
  14. Scoring function for $f_0$
  15. Scoring function for $f_1 \dots f_5$
  16. ...and 5 more sections

Figures (4)

  • Figure 1: Given an input image (a), the segmentation model $f$ outputs binary masks, (b)–(i) each representing a distinct semantic class.
  • Figure 2: A variety of weather and noise based (zoom_blur, glass_blur, snow, frost, fog and gaussian_noise) perturbations have been utilized to generate an extended synthetic test dataset that tests the autonomous railway system for robustness against weather.
  • Figure 3: DAG representing the topology of the Vision System
  • Figure 4: Response of the Vision System to various image corruption techniques.