Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mapping LLM Security Landscapes: A Comprehensive Stakeholder Risk Assessment Proposal

Rahul Pankajakshan, Sumitra Biswal, Yuvaraj Govindarajulu, Gilad Gressel

TL;DR

This paper addresses the lack of practical, stakeholder-focused risk assessment methods for LLM security by adapting the OWASP Risk Rating Methodology to LLM-based systems. It combines scenario analysis, dependency mapping, and impact analysis to produce a threat matrix that maps OWASP Top 10 LLM risks to three stakeholder groups: fine-tuning developers, API-integrating developers, and end users, and demonstrates the approach with a University Virtual Assistant use case. The study identifies prompts-injection as a high-risk threat and training-data poisoning as medium-risk, illustrating how mitigation strategies can be prioritized through the risk matrix and process. The proposed framework provides security practitioners with an actionable tool for resource allocation and risk mitigation planning, while highlighting the need for real-world validations and continuous refinement of threat models as LLM ecosystems evolve.

Abstract

The rapid integration of Large Language Models (LLMs) across diverse sectors has marked a transformative era, showcasing remarkable capabilities in text generation and problem-solving tasks. However, this technological advancement is accompanied by significant risks and vulnerabilities. Despite ongoing security enhancements, attackers persistently exploit these weaknesses, casting doubts on the overall trustworthiness of LLMs. Compounding the issue, organisations are deploying LLM-integrated systems without understanding the severity of potential consequences. Existing studies by OWASP and MITRE offer a general overview of threats and vulnerabilities but lack a method for directly and succinctly analysing the risks for security practitioners, developers, and key decision-makers who are working with this novel technology. To address this gap, we propose a risk assessment process using tools like the OWASP risk rating methodology which is used for traditional systems. We conduct scenario analysis to identify potential threat agents and map the dependent system components against vulnerability factors. Through this analysis, we assess the likelihood of a cyberattack. Subsequently, we conduct a thorough impact analysis to derive a comprehensive threat matrix. We also map threats against three key stakeholder groups: developers engaged in model fine-tuning, application developers utilizing third-party APIs, and end users. The proposed threat matrix provides a holistic evaluation of LLM-related risks, enabling stakeholders to make informed decisions for effective mitigation strategies. Our outlined process serves as an actionable and comprehensive tool for security practitioners, offering insights for resource management and enhancing the overall system security.

Mapping LLM Security Landscapes: A Comprehensive Stakeholder Risk Assessment Proposal

TL;DR

This paper addresses the lack of practical, stakeholder-focused risk assessment methods for LLM security by adapting the OWASP Risk Rating Methodology to LLM-based systems. It combines scenario analysis, dependency mapping, and impact analysis to produce a threat matrix that maps OWASP Top 10 LLM risks to three stakeholder groups: fine-tuning developers, API-integrating developers, and end users, and demonstrates the approach with a University Virtual Assistant use case. The study identifies prompts-injection as a high-risk threat and training-data poisoning as medium-risk, illustrating how mitigation strategies can be prioritized through the risk matrix and process. The proposed framework provides security practitioners with an actionable tool for resource allocation and risk mitigation planning, while highlighting the need for real-world validations and continuous refinement of threat models as LLM ecosystems evolve.

Abstract

The rapid integration of Large Language Models (LLMs) across diverse sectors has marked a transformative era, showcasing remarkable capabilities in text generation and problem-solving tasks. However, this technological advancement is accompanied by significant risks and vulnerabilities. Despite ongoing security enhancements, attackers persistently exploit these weaknesses, casting doubts on the overall trustworthiness of LLMs. Compounding the issue, organisations are deploying LLM-integrated systems without understanding the severity of potential consequences. Existing studies by OWASP and MITRE offer a general overview of threats and vulnerabilities but lack a method for directly and succinctly analysing the risks for security practitioners, developers, and key decision-makers who are working with this novel technology. To address this gap, we propose a risk assessment process using tools like the OWASP risk rating methodology which is used for traditional systems. We conduct scenario analysis to identify potential threat agents and map the dependent system components against vulnerability factors. Through this analysis, we assess the likelihood of a cyberattack. Subsequently, we conduct a thorough impact analysis to derive a comprehensive threat matrix. We also map threats against three key stakeholder groups: developers engaged in model fine-tuning, application developers utilizing third-party APIs, and end users. The proposed threat matrix provides a holistic evaluation of LLM-related risks, enabling stakeholders to make informed decisions for effective mitigation strategies. Our outlined process serves as an actionable and comprehensive tool for security practitioners, offering insights for resource management and enhancing the overall system security.
Paper Structure (34 sections, 1 equation, 1 figure, 4 tables)

This paper contains 34 sections, 1 equation, 1 figure, 4 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Rise of LLMs
  4. Risk Assessment
  5. OWASP Top 10 for LLM
  6. Risk Rating
  7. Likelihood factors
  8. Impact factors
  9. LLM Risks
  10. LLM01: Prompt Injection
  11. LLM02: Insecure Output Handling
  12. LLM03: Training Data Poisoning
  13. LLM04: Model Denial of Service
  14. LLM05: Supply Chain Vulnerabilities
  15. LLM06: Sensitive Information Disclosure
  16. ...and 19 more sections

Figures (1)

  • Figure 1: The system design of the university virtual assistant emphasizes the diverse security measures implemented for each component. An authenticated user's prompt undergoes optimization and validation/filtering for prompt injection within the prompt optimization module. Contextual information for the query is accessed based on authorization. The query and context serve as inputs to the LLM, and the resulting response from the LLM is logged and cached. Before being sent to the user, the response undergoes filtering. A rate-limiting mechanism has been implemented at the Gateway to safeguard against DoS attacks.