Assessing the Software Security Comprehension of Large Language Models

TL;DR

This work presents Basket, a Bloom’s taxonomy–guided framework for evaluating software-security knowledge in five leading LLMs. By integrating MCQs, vulnerable-code datasets (SALLM), project-based tasks, and real-world benchmarks (XBOW), the authors measure six cognitive levels from Remembering to Creating and estimate a software-security knowledge boundary $K_B(m)$ across Bloom levels with varying reliability thresholds. They also derive a taxonomy of 51 recurring misconceptions in LLMs’ software-security reasoning and provide a replication package to enable reproducibility. The study finds strong performance on low-level tasks but substantial degradation on higher-order tasks like evaluation and creation, with GPT-5-Mini and Gemini-2.5-Flash generally leading across many tasks and GPT-5-Mini achieving the broadest knowledge boundary. The findings have important implications for educators, researchers, and practitioners, highlighting both the educational utility of LLMs as scaffolds and the need for careful boundary-aware evaluation to mitigate misinterpretation and mis teaching in secure software development.

Abstract

Large language models (LLMs) are increasingly used in software development, but their level of software security expertise remains unclear. This work systematically evaluates the security comprehension of five leading LLMs: GPT-4o-Mini, GPT-5-Mini, Gemini-2.5-Flash, Llama-3.1, and Qwen-2.5, using Blooms Taxonomy as a framework. We assess six cognitive dimensions: remembering, understanding, applying, analyzing, evaluating, and creating. Our methodology integrates diverse datasets, including curated multiple-choice questions, vulnerable code snippets (SALLM), course assessments from an Introduction to Software Security course, real-world case studies (XBOW), and project-based creation tasks from a Secure Software Engineering course. Results show that while LLMs perform well on lower-level cognitive tasks such as recalling facts and identifying known vulnerabilities, their performance degrades significantly on higher-order tasks that require reasoning, architectural evaluation, and secure system creation. Beyond reporting aggregate accuracy, we introduce a software security knowledge boundary that identifies the highest cognitive level at which a model consistently maintains reliable performance. In addition, we identify 51 recurring misconception patterns exhibited by LLMs across Blooms levels.

Figures (7)

• Figure 1: Overview of BASKET (a Bloom’s Taxonomy-Guided Framework for Evaluating Software Security Knowledge).
• Figure 2: Prompt Used for the Models for Answering SQ1
• Figure 3: Prompt Used for the Models for Answering SQ2
• Figure 4: Prompt template used to answer SQ5 (case study).
• Figure 5: Prompt template used to answer SQ6 (Creating)