Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Llama-3.1-FoundationAI-SecurityLLM-Reasoning-8B Technical Report

Zhuoran Yang, Ed Li, Jianliang He, Aman Priyanshu, Baturay Saglam, Paul Kassianik, Sajana Weerawardhena, Anu Vellore, Blaine Nelson, Neusha Javidnia, Arthur Goldblatt, Fraser Burch, Avi Zohary, Assaf Eisenman, Mahdi Sabbaghi, Supriti Vijay, Rahim Dharssi, Dhruv Kedia, Kojin Oshiba, Yaron Singer, Amin Karbasi

TL;DR

Foundation-Sec-8B-Reasoning introduces an open-source 8B cybersecurity native reasoning model trained with supervised fine-tuning and reinforcement learning from verifiable rewards to generate explicit reasoning traces before answers. It uses a diverse ~2M exemplar SFT dataset across cybersecurity, mathematics, and coding, and addresses data heterogeneity and reward hacking with per-sample loss and a format penalty. The model achieves competitive cybersecurity performance, matching or approaching much larger models on critical benchmarks (e.g., CTIBench-RCM) while preserving broad general capabilities like AlpacaEval 2 and 2WikiMultihopQA. Safety is enhanced with system prompts and Llama-Guard style guardrails, and the model is released publicly, enabling research and deployment in security workflows.

Abstract

We present Foundation-Sec-8B-Reasoning, the first open-source native reasoning model for cybersecurity. Built upon our previously released Foundation-Sec-8B base model (derived from Llama-3.1-8B-Base), the model is trained through a two-stage process combining supervised fine-tuning (SFT) and reinforcement learning from verifiable rewards (RLVR). Our training leverages proprietary reasoning data spanning cybersecurity analysis, instruction-following, and mathematical reasoning. Evaluation across 10 cybersecurity benchmarks and 10 general-purpose benchmarks demonstrates performance competitive with significantly larger models on cybersecurity tasks while maintaining strong general capabilities. The model shows effective generalization on multi-hop reasoning tasks and strong safety performance when deployed with appropriate system prompts and guardrails. This work demonstrates that domain-specialized reasoning models can achieve strong performance on specialized tasks while maintaining broad general capabilities. We release the model publicly at https://huggingface.co/fdtn-ai/Foundation-Sec-8B-Reasoning.

Llama-3.1-FoundationAI-SecurityLLM-Reasoning-8B Technical Report

TL;DR

Foundation-Sec-8B-Reasoning introduces an open-source 8B cybersecurity native reasoning model trained with supervised fine-tuning and reinforcement learning from verifiable rewards to generate explicit reasoning traces before answers. It uses a diverse ~2M exemplar SFT dataset across cybersecurity, mathematics, and coding, and addresses data heterogeneity and reward hacking with per-sample loss and a format penalty. The model achieves competitive cybersecurity performance, matching or approaching much larger models on critical benchmarks (e.g., CTIBench-RCM) while preserving broad general capabilities like AlpacaEval 2 and 2WikiMultihopQA. Safety is enhanced with system prompts and Llama-Guard style guardrails, and the model is released publicly, enabling research and deployment in security workflows.

Abstract

We present Foundation-Sec-8B-Reasoning, the first open-source native reasoning model for cybersecurity. Built upon our previously released Foundation-Sec-8B base model (derived from Llama-3.1-8B-Base), the model is trained through a two-stage process combining supervised fine-tuning (SFT) and reinforcement learning from verifiable rewards (RLVR). Our training leverages proprietary reasoning data spanning cybersecurity analysis, instruction-following, and mathematical reasoning. Evaluation across 10 cybersecurity benchmarks and 10 general-purpose benchmarks demonstrates performance competitive with significantly larger models on cybersecurity tasks while maintaining strong general capabilities. The model shows effective generalization on multi-hop reasoning tasks and strong safety performance when deployed with appropriate system prompts and guardrails. This work demonstrates that domain-specialized reasoning models can achieve strong performance on specialized tasks while maintaining broad general capabilities. We release the model publicly at https://huggingface.co/fdtn-ai/Foundation-Sec-8B-Reasoning.
Paper Structure (48 sections, 5 figures, 7 tables)

This paper contains 48 sections, 5 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Open-Source Reasoning Models.
  4. Cybersecurity LLMs.
  5. Methodology
  6. Stage 1: Supervised Fine-Tuning for Native Reasoning
  7. SFT Dataset and Training
  8. Stage 2: Reinforcement Learning with Verifiable Rewards
  9. Data Heterogeneity.
  10. Reward Hacking and Format Degradation.
  11. Evaluation Results
  12. Cybersecurity Benchmarks
  13. Benchmarks.
  14. Models.
  15. Evaluation Protocol.
  16. ...and 33 more sections

Figures (5)

  • Figure 1: Performance comparison of Foundation-Sec-8B-Reasoning against baseline models. (a) On cybersecurity benchmarks (CTIBench-MCQA, CTIBench-RCM, CTI-Reasoning, CWE-Prediction, SecBench, SecEval), our model performs on par with the 70B model (Llama-3.3-70B-Instruct) while significantly outperforming our previous instruction-tuned model (Foundation-Sec-8B-Instruct). (b) On general-purpose benchmarks (AlpacaEval 2, BBH, IFEval, GSM8K, HumanEval, MATH), our model achieves comparable performance to Llama-3.1-8B-Instruct on most tasks, with significantly better performance on AlpacaEval 2.
  • Figure 2: Data composition for the SFT and RL training stages. The SFT stage uses a diverse mix of data to instill broad reasoning abilities, while the RL stage emphasizes instruction following, cybersecurity, and mathematical reasoning to further refine the model's reasoning abilities.
  • Figure 3: Comparison of selected models across 6 key cybersecurity benchmarks. The benchmarks are organized in two rows: (top row) CTIBench-MCQA, CTIBench-RCM, and CTI-Reasoning; (bottom row) CWE-Prediction, SecBench-Reasoning, and SecEval. Foundation-Sec-8B-Reasoning demonstrates consistently strong performance across diverse tasks, particularly excelling on CTIBench-RCM (75.3%) and CWE-Prediction (70.4%). The visualization highlights the model's competitive performance with significantly larger models while maintaining robust capabilities across both knowledge-based and reasoning-intensive benchmarks.
  • Figure 4: Comparison of model performance across 6 key general-purpose benchmarks. The benchmarks are organized in two rows: (top row) AlpacaEval 2 (human preference), BBH (reasoning), and IFEval (instruction following); (bottom row) GSM8K (grade school math), HumanEval (coding), and MATH (competition mathematics). Error bars represent one standard deviation. Foundation-Sec-8B-Reasoning (highlighted with hatched pattern) demonstrates strong performance across diverse capabilities, with exceptional results on AlpacaEval 2 (62.6%) and competitive performance on reasoning and coding tasks.
  • Figure 5: HarmBench safety evaluation results showing pass rates (percentage of harmful prompts appropriately refused) across different model configurations. Foundation-Sec-8B-Instruct achieves 95.00% pass rate, while Foundation-Sec-8B-Reasoning achieves 93.00% with proper system prompts. When further protected by Llama-Guard-3-8B, Foundation-Sec-8B-Reasoning achieves 98.25% pass rate, demonstrating that our reasoning model with appropriate safety measures delivers strong protection against adversarial prompts.