Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MoRSE: Bridging the Gap in Cybersecurity Expertise with Retrieval Augmented Generation

Marco Simoni, Andrea Saracino, Vinod P., Mauro Conti

TL;DR

MoRSE introduces a novel dual-RAG architecture (Structured and Unstructured) that leverages non-parametric knowledge bases and real-time updates to deliver accurate cybersecurity Q&A. Instead of relying solely on parametric memory, MoRSE retrieves diverse, multidimensional data from domain sources (MITRE, Metasploit, ExploitDB, CWE) and composes context-aware answers via an LLM, with an emphasis on handling multi-hop and CVE/CWE queries. The paper presents a three-part evaluation framework, plus LLM-as-Judge analyses, showing MoRSE outperforms leading LLMs like GPT-4 and Mixtral in relevance, correctness, and CVE-identification accuracy by significant margins. Key contributions include open-source implementation, a cascade RAG design enabling continuous knowledge enrichment, and detailed retriever impact analyses that guide future optimizations. The work has practical implications for threat intelligence, incident response, and cybersecurity education, and outlines future directions such as an enhanced knowledge graph and smarter caching to further improve performance and scalability.

Abstract

In this paper, we introduce MoRSE (Mixture of RAGs Security Experts), the first specialised AI chatbot for cybersecurity. MoRSE aims to provide comprehensive and complete knowledge about cybersecurity. MoRSE uses two RAG (Retrieval Augmented Generation) systems designed to retrieve and organize information from multidimensional cybersecurity contexts. MoRSE differs from traditional RAGs by using parallel retrievers that work together to retrieve semantically related information in different formats and structures. Unlike traditional Large Language Models (LLMs) that rely on Parametric Knowledge Bases, MoRSE retrieves relevant documents from Non-Parametric Knowledge Bases in response to user queries. Subsequently, MoRSE uses this information to generate accurate answers. In addition, MoRSE benefits from real-time updates to its knowledge bases, enabling continuous knowledge enrichment without retraining. We have evaluated the effectiveness of MoRSE against other state-of-the-art LLMs, evaluating the system on 600 cybersecurity specific questions. The experimental evaluation has shown that the improvement in terms of relevance and correctness of the answer is more than 10\% compared to known solutions such as GPT-4 and Mixtral 7x8.

MoRSE: Bridging the Gap in Cybersecurity Expertise with Retrieval Augmented Generation

TL;DR

MoRSE introduces a novel dual-RAG architecture (Structured and Unstructured) that leverages non-parametric knowledge bases and real-time updates to deliver accurate cybersecurity Q&A. Instead of relying solely on parametric memory, MoRSE retrieves diverse, multidimensional data from domain sources (MITRE, Metasploit, ExploitDB, CWE) and composes context-aware answers via an LLM, with an emphasis on handling multi-hop and CVE/CWE queries. The paper presents a three-part evaluation framework, plus LLM-as-Judge analyses, showing MoRSE outperforms leading LLMs like GPT-4 and Mixtral in relevance, correctness, and CVE-identification accuracy by significant margins. Key contributions include open-source implementation, a cascade RAG design enabling continuous knowledge enrichment, and detailed retriever impact analyses that guide future optimizations. The work has practical implications for threat intelligence, incident response, and cybersecurity education, and outlines future directions such as an enhanced knowledge graph and smarter caching to further improve performance and scalability.

Abstract

In this paper, we introduce MoRSE (Mixture of RAGs Security Experts), the first specialised AI chatbot for cybersecurity. MoRSE aims to provide comprehensive and complete knowledge about cybersecurity. MoRSE uses two RAG (Retrieval Augmented Generation) systems designed to retrieve and organize information from multidimensional cybersecurity contexts. MoRSE differs from traditional RAGs by using parallel retrievers that work together to retrieve semantically related information in different formats and structures. Unlike traditional Large Language Models (LLMs) that rely on Parametric Knowledge Bases, MoRSE retrieves relevant documents from Non-Parametric Knowledge Bases in response to user queries. Subsequently, MoRSE uses this information to generate accurate answers. In addition, MoRSE benefits from real-time updates to its knowledge bases, enabling continuous knowledge enrichment without retraining. We have evaluated the effectiveness of MoRSE against other state-of-the-art LLMs, evaluating the system on 600 cybersecurity specific questions. The experimental evaluation has shown that the improvement in terms of relevance and correctness of the answer is more than 10\% compared to known solutions such as GPT-4 and Mixtral 7x8.
Paper Structure (45 sections, 4 equations, 6 figures, 7 tables, 2 algorithms)

This paper contains 45 sections, 4 equations, 6 figures, 7 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Contribution
  3. Organisation
  4. background
  5. Large Language Models
  6. Retrieval Augmented Generation
  7. Definitions
  8. MoRSE Architecture
  9. MoRSE Overview
  10. MoRSE Core
  11. MoRSE Core Workflow
  12. RAG Architecture
  13. Query Handling
  14. Functionalities
  15. Components
  16. ...and 30 more sections

Figures (6)

  • Figure 1: MoRSE Overview.
  • Figure 2: MoRSE Core Workflow.
  • Figure 3: RAG Architecture Overview.
  • Figure 4: Structured RAG Workflow.
  • Figure 5: Unstructured RAG Workflow
  • ...and 1 more figures