Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Designing Secure AI-based Systems: a Multi-Vocal Literature Review

Simon Schneider, Ananya Saha, Emanuele Mezzi, Katja Tuma, Riccardo Scandariato

TL;DR

The paper tackles the lack of concrete architectural security guidance for AI-based systems by conducting a multi-vocal literature review that identifies 16 actionable guidelines (G1–G16) and maps them to common AI-system components. It combines academic SLR methods with gray-literature exploration, though the gray-literature study yields a negative result, underscoring a gap in practitioner guidance. The primary contributions are the curated guideline set, a transparent mapping to AI-system components, and a replication package, providing practitioners with concrete design principles while highlighting research gaps and standards needs. Overall, the work advances the practical design of secure AI-based architectures and motivates further empirical and normative work in this area.

Abstract

AI-based systems leverage recent advances in the field of AI/ML by combining traditional software systems with AI components. Applications are increasingly being developed in this way. Software engineers can usually rely on a plethora of supporting information on how to use and implement any given technology. For AI-based systems, however, such information is scarce. Specifically, guidance on how to securely design the architecture is not available to the extent as for other systems. We present 16 architectural security guidelines for the design of AI-based systems that were curated via a multi-vocal literature review. The guidelines could support practitioners with actionable advice on the secure development of AI-based systems. Further, we mapped the guidelines to typical components of AI-based systems and observed a high coverage where 6 out of 8 generic components have at least one guideline associated to them.

Designing Secure AI-based Systems: a Multi-Vocal Literature Review

TL;DR

The paper tackles the lack of concrete architectural security guidance for AI-based systems by conducting a multi-vocal literature review that identifies 16 actionable guidelines (G1–G16) and maps them to common AI-system components. It combines academic SLR methods with gray-literature exploration, though the gray-literature study yields a negative result, underscoring a gap in practitioner guidance. The primary contributions are the curated guideline set, a transparent mapping to AI-system components, and a replication package, providing practitioners with concrete design principles while highlighting research gaps and standards needs. Overall, the work advances the practical design of secure AI-based architectures and motivates further empirical and normative work in this area.

Abstract

AI-based systems leverage recent advances in the field of AI/ML by combining traditional software systems with AI components. Applications are increasingly being developed in this way. Software engineers can usually rely on a plethora of supporting information on how to use and implement any given technology. For AI-based systems, however, such information is scarce. Specifically, guidance on how to securely design the architecture is not available to the extent as for other systems. We present 16 architectural security guidelines for the design of AI-based systems that were curated via a multi-vocal literature review. The guidelines could support practitioners with actionable advice on the secure development of AI-based systems. Further, we mapped the guidelines to typical components of AI-based systems and observed a high coverage where 6 out of 8 generic components have at least one guideline associated to them.
Paper Structure (10 sections, 3 figures, 4 tables)

This paper contains 10 sections, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Systematic Review of Academic Literature
  3. Selection of Papers
  4. Curation of Architectural Security Guidelines
  5. Gray Literature Study with Negative Results
  6. Guidelines Curated from Literature (RQ1)
  7. Mapping to Typical Components of AI-based Systems (RQ2)
  8. Threats to Validity
  9. Related Work
  10. Research Outlook

Figures (3)

  • Figure 1: Methodology of the conducted SLR and extraction of guidelines.
  • Figure 2: Methodology of gray literature study with negative results.
  • Figure 3: Mapping of architectural security guidelines to typical components of AI-based systems, adapted from Kästner Kastner22_ml_in_production.