Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Large Language Models for Explainable Threat Intelligence

Tiago Dinis, Miguel Correia, Roger Tavares

TL;DR

The paper tackles the need for timely, explainable cyber threat intelligence (CTI) by integrating large language models with retrieval-augmented generation and per-answer knowledge graphs. It introduces RAGRecon, an architecture that retrieves context from domain data, grounds LLM reasoning, and visualizes relationships as a knowledge graph to improve transparency. Through experiments on conventional and blockchain CTI datasets with seven LLMs, the approach achieves high faithfulness (above 0.8) and meaningful context usage (about 8%), with best configurations matching reference answers in over 91% of cases. The work demonstrates a practical, explainable CTI pipeline for SOC analysts and blockchain security, while identifying limitations in KG reliability for smaller models and highlighting directions for broader applicability and robustness.

Abstract

As cyber threats continue to grow in complexity, traditional security mechanisms struggle to keep up. Large language models (LLMs) offer significant potential in cybersecurity due to their advanced capabilities in text processing and generation. This paper explores the use of LLMs with retrieval-augmented generation (RAG) to obtain threat intelligence by combining real-time information retrieval with domain-specific data. The proposed system, RAGRecon, uses a LLM with RAG to answer questions about cybersecurity threats. Moreover, it makes this form of Artificial Intelligence (AI) explainable by generating and visually presenting to the user a knowledge graph for every reply. This increases the transparency and interpretability of the reasoning of the model, allowing analysts to better understand the connections made by the system based on the context recovered by the RAG system. We evaluated RAGRecon experimentally with two datasets and seven different LLMs and the responses matched the reference responses more than 91% of the time for the best combinations.

Large Language Models for Explainable Threat Intelligence

TL;DR

The paper tackles the need for timely, explainable cyber threat intelligence (CTI) by integrating large language models with retrieval-augmented generation and per-answer knowledge graphs. It introduces RAGRecon, an architecture that retrieves context from domain data, grounds LLM reasoning, and visualizes relationships as a knowledge graph to improve transparency. Through experiments on conventional and blockchain CTI datasets with seven LLMs, the approach achieves high faithfulness (above 0.8) and meaningful context usage (about 8%), with best configurations matching reference answers in over 91% of cases. The work demonstrates a practical, explainable CTI pipeline for SOC analysts and blockchain security, while identifying limitations in KG reliability for smaller models and highlighting directions for broader applicability and robustness.

Abstract

As cyber threats continue to grow in complexity, traditional security mechanisms struggle to keep up. Large language models (LLMs) offer significant potential in cybersecurity due to their advanced capabilities in text processing and generation. This paper explores the use of LLMs with retrieval-augmented generation (RAG) to obtain threat intelligence by combining real-time information retrieval with domain-specific data. The proposed system, RAGRecon, uses a LLM with RAG to answer questions about cybersecurity threats. Moreover, it makes this form of Artificial Intelligence (AI) explainable by generating and visually presenting to the user a knowledge graph for every reply. This increases the transparency and interpretability of the reasoning of the model, allowing analysts to better understand the connections made by the system based on the context recovered by the RAG system. We evaluated RAGRecon experimentally with two datasets and seven different LLMs and the responses matched the reference responses more than 91% of the time for the best combinations.

Paper Structure

This paper contains 40 sections, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Large Language Models
  4. Retrieval-Augmented Generation
  5. Cyber Threat Intelligence
  6. Enhancing CTI with LLMs and RAG
  7. RAGRecon
  8. Architecture
  9. Data Ingestion and Vectorization
  10. Interactive RAG
  11. Textual Response Generation
  12. Knowledge Graph Generation and Visualization
  13. Implementation of RAGRecon
  14. Data Ingestion and Vectorization
  15. Document Loading and Pre-processing
  16. ...and 25 more sections

Figures (6)

  • Figure 1: Data flow diagram of the proposed system architecture
  • Figure 2: Data flow diagram with the technologies used in the architecture
  • Figure 3: Knowledge Graph Example
  • Figure 4: Comparison different models Round 3 Conventional CTI Dataset
  • Figure 5: RAG answer match reference answer Conventional CTI Dataset
  • ...and 1 more figures