Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLMs' Suitability for Network Security: A Case Study of STRIDE Threat Modeling

AbdulAziz AbdulGhaffar, Ashraf Matrawy

TL;DR

The paper assesses the suitability of Large Language Models (LLMs) for network-security threat modeling using the STRIDE framework in a 5G context. By evaluating four prompting strategies across five LLMs on six predefined threats, it reveals that prompting with examples improves STRIDE classification accuracy (63% to ~71%) and F1 (~52% to ~62%), but many second-order threats and perspective issues remain challenging. The study highlights that no single model dominates, and performance differences are sensitive to prompting approach; it advocates telecom-specific fine-tuning to achieve robust threat-modeling in AI-native, future networks. Overall, the work provides actionable insights into prompting design and the limitations of off-the-shelf LLMs for automated network threat classification, guiding future optimization and integration in security workflows.

Abstract

Artificial Intelligence (AI) is expected to be an integral part of next-generation AI-native 6G networks. With the prevalence of AI, researchers have identified numerous use cases of AI in network security. However, there are very few studies that analyze the suitability of Large Language Models (LLMs) in network security. To fill this gap, we examine the suitability of LLMs in network security, particularly with the case study of STRIDE threat modeling. We utilize four prompting techniques with five LLMs to perform STRIDE classification of 5G threats. From our evaluation results, we point out key findings and detailed insights along with the explanation of the possible underlying factors influencing the behavior of LLMs in the modeling of certain threats. The numerical results and the insights support the necessity for adjusting and fine-tuning LLMs for network security use cases.

LLMs' Suitability for Network Security: A Case Study of STRIDE Threat Modeling

TL;DR

The paper assesses the suitability of Large Language Models (LLMs) for network-security threat modeling using the STRIDE framework in a 5G context. By evaluating four prompting strategies across five LLMs on six predefined threats, it reveals that prompting with examples improves STRIDE classification accuracy (63% to ~71%) and F1 (~52% to ~62%), but many second-order threats and perspective issues remain challenging. The study highlights that no single model dominates, and performance differences are sensitive to prompting approach; it advocates telecom-specific fine-tuning to achieve robust threat-modeling in AI-native, future networks. Overall, the work provides actionable insights into prompting design and the limitations of off-the-shelf LLMs for automated network threat classification, guiding future optimization and integration in security workflows.

Abstract

Artificial Intelligence (AI) is expected to be an integral part of next-generation AI-native 6G networks. With the prevalence of AI, researchers have identified numerous use cases of AI in network security. However, there are very few studies that analyze the suitability of Large Language Models (LLMs) in network security. To fill this gap, we examine the suitability of LLMs in network security, particularly with the case study of STRIDE threat modeling. We utilize four prompting techniques with five LLMs to perform STRIDE classification of 5G threats. From our evaluation results, we point out key findings and detailed insights along with the explanation of the possible underlying factors influencing the behavior of LLMs in the modeling of certain threats. The numerical results and the insights support the necessity for adjusting and fine-tuning LLMs for network security use cases.
Paper Structure (12 sections, 4 figures, 1 table)

This paper contains 12 sections, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Motivation and Examination of Related Work
  3. Evaluation Methodology for A Case Study of stride Threat Modeling
  4. Selected Threats and Vulnerabilities
  5. Large-Language Models (LLMs)
  6. Prompts
  7. Results and Insights
  8. -based Classification
  9. Insights on the -based Classification
  10. Comparison of Prompting Techniques
  11. Performance of in Classification
  12. Discussion and Conclusion

Figures (4)

  • Figure 1: Evaluation Methodology of stride Modeling
  • Figure 2: Accuracy of LLMs in 5G STRIDE Modeling using different prompting techniques
  • Figure 3: F1 Score of LLMs in 5G STRIDE Modeling using different prompting techniques
  • Figure 4: Heatmap showing the performance of in terms of accuracy, precision, recall, and F1 score. Higher values (dark green color) are better.