Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AI Safety Evaluations Need To Consider Cascading Effects

Anna Neumann, Jatinder Singh

TL;DR

This paper introduces the cascade as a concept for supporting more holistic AI evaluations, and advocates for a paradigm shift in AI system auditing towards systems-oriented audits that incorporate cascading effects, to complement model centric evaluations.

Abstract

AI systems comprise a range of interactions across the technical and organisational components of a range of actors. These components work together to provide the systems' functionality. This socio-technical assemblage is increasingly described as an algorithmic supply chain. Given their role in supporting a wide range of systems, foundation models (FMs) are increasingly a key part of many algorithmic supply chains. In practice, various technical and non-technical components work to mediate, adapt, and augment the behaviour of models, such as FMs, both in general, and for their use in specific application contexts. However, many AI safety evaluations tend to focus on capabilities of FMs themselves and/or assess these components independently, at certain levels of abstraction, with less consideration on how these components could interact, influence, reinforce or counteract each other. In this paper, we introduce the cascade as a concept for supporting more holistic AI evaluations. The cascade captures how the interactions between socio-technical components along the AI supply chain can compound and produce cumulative effects with downstream consequences. Specifically, we (i) identify gaps in current AI auditing approaches, using LLMs as our case study; (ii) demonstrate how cascade problems manifest in deployed AI systems through a characterisation of cascades through different viewpoints (component and stakeholder), and (iii) propose research directions for assessing the cascade specifically as an object of analysis. As these cascades can significantly impact transparency, accountability, security, and safety, we advocate for a paradigm shift in AI system auditing towards systems-oriented audits that incorporate cascading effects, to complement model centric evaluations.

AI Safety Evaluations Need To Consider Cascading Effects

TL;DR

This paper introduces the cascade as a concept for supporting more holistic AI evaluations, and advocates for a paradigm shift in AI system auditing towards systems-oriented audits that incorporate cascading effects, to complement model centric evaluations.

Abstract

AI systems comprise a range of interactions across the technical and organisational components of a range of actors. These components work together to provide the systems' functionality. This socio-technical assemblage is increasingly described as an algorithmic supply chain. Given their role in supporting a wide range of systems, foundation models (FMs) are increasingly a key part of many algorithmic supply chains. In practice, various technical and non-technical components work to mediate, adapt, and augment the behaviour of models, such as FMs, both in general, and for their use in specific application contexts. However, many AI safety evaluations tend to focus on capabilities of FMs themselves and/or assess these components independently, at certain levels of abstraction, with less consideration on how these components could interact, influence, reinforce or counteract each other. In this paper, we introduce the cascade as a concept for supporting more holistic AI evaluations. The cascade captures how the interactions between socio-technical components along the AI supply chain can compound and produce cumulative effects with downstream consequences. Specifically, we (i) identify gaps in current AI auditing approaches, using LLMs as our case study; (ii) demonstrate how cascade problems manifest in deployed AI systems through a characterisation of cascades through different viewpoints (component and stakeholder), and (iii) propose research directions for assessing the cascade specifically as an object of analysis. As these cascades can significantly impact transparency, accountability, security, and safety, we advocate for a paradigm shift in AI system auditing towards systems-oriented audits that incorporate cascading effects, to complement model centric evaluations.
Paper Structure (17 sections, 2 figures, 1 table)

This paper contains 17 sections, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Contributions
  3. Definitions
  4. Background: Seeing (through) AI Supply Chains
  5. The Cascade
  6. Component Cascades
  7. Stakeholder Cascade
  8. The Current Foundation Model AI Auditing Landscape
  9. Foundation Model Audits
  10. Application Audits
  11. Auditing Gaps
  12. Approaches to Cascade Auditing
  13. Technical Research Directions
  14. Human-Centered and Interdisciplinary Approaches
  15. Tensions and Resulting Research Opportunities
  16. ...and 2 more sections

Figures (2)

  • Figure 1: A conceptual illustration an AI supply chain depicting the interactions between a FM and a user-facing application ('deployed system') between which we see multiple components influenced by different stakeholders: here the model provider, application developer, and user.
  • Figure 2: An overall view of possible implementations of cascades through AI supply chains. Orange (user request path) and green (response path) signal different component combinations. Illustrating the dynamic nature of the interactions, the AIaaS results (green and red symbol) directly effect what kind of component gets involved when processing a user request as the following step.