Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RedChronos: A Large Language Model-Based Log Analysis System for Insider Threat Detection in Enterprises

Chenyu Li, Zhengjia Zhu, Jiyan He, Xiu Zhang

TL;DR

RedChronos addresses the challenge of insider threat detection in enterprise log data by integrating a dual-LLM framework: Query-Aware Weighted Voting (QAWV) for robust ensemble decisions and a Semantic Expansion-based Genetic Algorithm (SEGA) for automatic prompt evolution. The system architecture (Pipeline System, LLM Gate, Model Dispatch, Database, Audit System) enables scalable ingestion, analysis, and SOC-oriented triage, while leveraging open- and closed-source LLMs. On public CERT datasets (r4.2 and r5.2), RedChronos achieves high detection rates with very low false positives, outperforming or matching prior approaches and reducing SOC workload by about 90% in enterprise deployments. The work demonstrates the importance of prompt optimization and model-ensemble strategies for reliable LLM-based security analytics and suggests avenues for real-time adaptation and broader SOC integration. Overall, RedChronos offers a practical, scalable solution for automated log analysis and insider threat detection with tangible benefits for enterprise security operations.

Abstract

Internal threat detection (IDT) aims to address security threats within organizations or enterprises by identifying potential or already occurring malicious threats within vast amounts of logs. Although organizations or enterprises have dedicated personnel responsible for reviewing these logs, it is impossible to manually examine all logs entirely.In response to the vast number of logs, we propose a system called RedChronos, which is a Large Language Model-Based Log Analysis System. This system incorporates innovative improvements over previous research by employing Query-Aware Weighted Voting and a Semantic Expansion-based Genetic Algorithm with LLM-driven Mutations. On the public datasets CERT 4.2 and 5.2, RedChronos outperforms or matches existing approaches in terms of accuracy, precision, and detection rate. Moreover, RedChronos reduces the need for manual intervention in security log reviews by approximately 90% in the Xiaohongshu Security Operation Center. Therefore, our RedChronos system demonstrates exceptional performance in handling IDT tasks, providing innovative solutions for these challenges. We believe that future research can continue to enhance the system's performance in IDT tasks while also reducing the response time to internal risk events.

RedChronos: A Large Language Model-Based Log Analysis System for Insider Threat Detection in Enterprises

TL;DR

RedChronos addresses the challenge of insider threat detection in enterprise log data by integrating a dual-LLM framework: Query-Aware Weighted Voting (QAWV) for robust ensemble decisions and a Semantic Expansion-based Genetic Algorithm (SEGA) for automatic prompt evolution. The system architecture (Pipeline System, LLM Gate, Model Dispatch, Database, Audit System) enables scalable ingestion, analysis, and SOC-oriented triage, while leveraging open- and closed-source LLMs. On public CERT datasets (r4.2 and r5.2), RedChronos achieves high detection rates with very low false positives, outperforming or matching prior approaches and reducing SOC workload by about 90% in enterprise deployments. The work demonstrates the importance of prompt optimization and model-ensemble strategies for reliable LLM-based security analytics and suggests avenues for real-time adaptation and broader SOC integration. Overall, RedChronos offers a practical, scalable solution for automated log analysis and insider threat detection with tangible benefits for enterprise security operations.

Abstract

Internal threat detection (IDT) aims to address security threats within organizations or enterprises by identifying potential or already occurring malicious threats within vast amounts of logs. Although organizations or enterprises have dedicated personnel responsible for reviewing these logs, it is impossible to manually examine all logs entirely.In response to the vast number of logs, we propose a system called RedChronos, which is a Large Language Model-Based Log Analysis System. This system incorporates innovative improvements over previous research by employing Query-Aware Weighted Voting and a Semantic Expansion-based Genetic Algorithm with LLM-driven Mutations. On the public datasets CERT 4.2 and 5.2, RedChronos outperforms or matches existing approaches in terms of accuracy, precision, and detection rate. Moreover, RedChronos reduces the need for manual intervention in security log reviews by approximately 90% in the Xiaohongshu Security Operation Center. Therefore, our RedChronos system demonstrates exceptional performance in handling IDT tasks, providing innovative solutions for these challenges. We believe that future research can continue to enhance the system's performance in IDT tasks while also reducing the response time to internal risk events.

Paper Structure

This paper contains 21 sections, 3 figures, 8 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. System Design
  4. Pipeline System
  5. LLM Gate
  6. Query-Aware Weighted Voting
  7. Semantic Expansion-based Genetic Algorithm
  8. Model Dispatch
  9. Database
  10. Audit System
  11. Experimental Setup
  12. Research Questions and Experimental Configuration
  13. Dataset
  14. Results And Analysis
  15. Overall Performance
  16. ...and 6 more sections

Figures (3)

  • Figure 1: The overall system design of RedChronos features circles with numbers that represent the flow of information throughout the entire system when log entries are input.
  • Figure 2: Flow chart of the Semantic Expansion-based Genetic Algorithm, where circles 1 to 5 represent a single execution cycle of the algorithm.
  • Figure 3: We use the correct and incorrect results of seed prompts on the test set as example inputs for demonstration to the LLM. An instructional prompt is then employed to guide the LLM in modifying the seed prompt based on the seed prompt itself, along with the correct and incorrect results. Finally, we utilize the new prompt as the updated seed prompt and proceed to test it. The Prompt Variation based on Semantic Expansion algorithm is denoted as PvboSemanticExpan.