Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A LINDDUN-based Privacy Threat Modeling Framework for GenAI

Qianying Liao, Jonah Bellemans, Laurens Sion, Xue Jiang, Dmitrii Usynin, Xuebing Zhou, Dimitri Van Landuyt, Lieven Desmet, Wouter Joosen

TL;DR

A novel domain-specific privacy threat modeling framework to support the privacy threat analysis of GenAI-based applications is introduced and validated on an AI Agent system, which demonstrates that a comprehensive privacy analysis can be supported by the new framework.

Abstract

As generative AI (GenAI) systems become increasingly prevalent across various technological stacks, the question of how such systems handle sensitive and personal data flows becomes increasingly important. Specifically, both the ability to harness and process large swaths of information as well as their stochastic nature raise key concerns related to both security and privacy. Unfortunately, while some of the traditional security threat modeling can effectively identify certain violations, privacy-related issues are often overlooked. To respond to these challenges, we introduce a novel domain-specific privacy threat modeling framework to support the privacy threat analysis of GenAI-based applications. This framework is constructed through a two-pronged approach: (1) a systematic review of the emerging literature on GenAI privacy threats, and (2) a case-driven application to a representative Chatbot system. These efforts yield a foundational GenAI privacy threat modeling framework built on LINDDUN. The new framework affects three out of the seven privacy threat types of LINDDUN and introduces 100 new GenAI examples to the knowledge base. Its effectiveness is validated on an AI Agent system, which demonstrates that a comprehensive privacy analysis can be supported by the new framework.

A LINDDUN-based Privacy Threat Modeling Framework for GenAI

TL;DR

A novel domain-specific privacy threat modeling framework to support the privacy threat analysis of GenAI-based applications is introduced and validated on an AI Agent system, which demonstrates that a comprehensive privacy analysis can be supported by the new framework.

Abstract

As generative AI (GenAI) systems become increasingly prevalent across various technological stacks, the question of how such systems handle sensitive and personal data flows becomes increasingly important. Specifically, both the ability to harness and process large swaths of information as well as their stochastic nature raise key concerns related to both security and privacy. Unfortunately, while some of the traditional security threat modeling can effectively identify certain violations, privacy-related issues are often overlooked. To respond to these challenges, we introduce a novel domain-specific privacy threat modeling framework to support the privacy threat analysis of GenAI-based applications. This framework is constructed through a two-pronged approach: (1) a systematic review of the emerging literature on GenAI privacy threats, and (2) a case-driven application to a representative Chatbot system. These efforts yield a foundational GenAI privacy threat modeling framework built on LINDDUN. The new framework affects three out of the seven privacy threat types of LINDDUN and introduces 100 new GenAI examples to the knowledge base. Its effectiveness is validated on an AI Agent system, which demonstrates that a comprehensive privacy analysis can be supported by the new framework.
Paper Structure (60 sections, 6 figures, 2 tables)

This paper contains 60 sections, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Contribution statement.
  3. Background & related work
  4. Privacy threat modeling methodologies
  5. Security threat modeling.
  6. Privacy threat modeling.
  7. GenAI systems in practice
  8. Methodology
  9. Foundation framework selection
  10. Threat knowledge generation
  11. Top-down: systematic literature survey
  12. Privacy knowledge generation.
  13. Bottom-up: HR Chatbot case study
  14. Privacy knowledge generation.
  15. Consolidation
  16. ...and 45 more sections

Figures (6)

  • Figure 1: Overview of the approach.
  • Figure 2: GenAI-integrated systems.
  • Figure 3: Example unawareness & unintervenability tree. The 'U.2.2 Access' and 'U.2.3 Rectification/erasure' threat characteristics were split and refined into AI- and GenAI-specific sub-characteristics.
  • Figure 4: Domain metamodel and hierarchy. This diagram illustrates how the linddun threat knowledge metamodel (gray) is extended with a domain-class that can be referred to across the threat knowledge tree. The right-hand side shows how GenAI is encoded in the domain hierarchy.
  • Figure 5: PRISMA diagram outlining the search protocol
  • ...and 1 more figures