Asset-centric Threat Modeling for AI-based Systems
Jan von der Assen, Jamo Sharif, Chao Feng, Christian Killer, Gérôme Bovet, Burkhard Stiller
TL;DR
AI threat modeling has lacked practical automation for AI-centric systems. This paper introduces ThreatFinderAI, an asset-centric threat modeling approach and tool that integrates a knowledge-graph of AI threats, an asset stencil library, and Monte Carlo risk quantification to model AI assets and residual risk. It aligns threat modeling with the AI lifecycle and enterprise risk management to identify threats to AI assets, propose controls, and quantify residual risk. Through a scenario-driven field experiment recreating expert models and a case study on an LLM-based legal workflow, the work demonstrates that ThreatFinderAI guides threat identification, supports risk discussion, and is usable by non-security experts in design-time security assessment.
Abstract
Threat modeling is a popular method to securely develop systems by achieving awareness of potential areas of future damage caused by adversaries. However, threat modeling for systems relying on Artificial Intelligence is still not well explored. While conventional threat modeling methods and tools did not address AI-related threats, research on this amalgamation still lacks solutions capable of guiding and automating the process, as well as providing evidence that the methods hold up in practice. Consequently, this paper presents ThreatFinderAI, an approach and tool providing guidance and automation to model AI-related assets, threats, countermeasures, and quantify residual risks. To evaluate the practicality of the approach, participants were tasked to recreate a threat model developed by cybersecurity experts of an AI-based healthcare platform. Secondly, the approach was used to identify and discuss strategic risks in an LLM-based application through a case study. Overall, the solution's usability was well-perceived and effectively supports threat identification and risk discussion.