RuleGenie: SIEM Detection Rule Set Optimization

TL;DR

RuleGenie targets SIEM rule-set inefficiency by combining CodeT5-based rule embeddings, cosine similarity-based top-$k$ rule retrieval, and LLM-assisted semantic analysis with chain-of-thought reasoning to identify redundancies and propose refinements. The methodology emphasizes platform-agnostic applicability across Sigma, Splunk, and AQL formats, supported by a rigorous ablation study that selects $k=5$ and a $75$ similarity threshold, and a cost-aware LLM choice (Qwen-2.5-14B-Instruct) for on-prem deployment. Key contributions include a novel embedding-based redundancy detection pipeline, a detailed LLM-driven evaluation with hierarchical dependence mapping, and actionable rule-recommendation strategies (keep, merge, or keep both). The results show improved precision and recall in redundancy detection and rule-set optimization, with substantial efficiency gains and practical benefits for SOC teams in enterprise SIEM environments.

Abstract

SIEM systems serve as a critical hub, employing rule-based logic to detect and respond to threats. Redundant or overlapping rules in SIEM systems lead to excessive false alerts, degrading analyst performance due to alert fatigue, and increase computational overhead and response latency for actual threats. As a result, optimizing SIEM rule sets is essential for efficient operations. Despite the importance of such optimization, research in this area is limited, with current practices relying on manual optimization methods that are both time-consuming and error-prone due to the scale and complexity of enterprise-level rule sets. To address this gap, we present RuleGenie, a novel large language model (LLM) aided recommender system designed to optimize SIEM rule sets. Our approach leverages transformer models' multi-head attention capabilities to generate SIEM rule embeddings, which are then analyzed using a similarity matching algorithm to identify the top-k most similar rules. The LLM then processes the rules identified, utilizing its information extraction, language understanding, and reasoning capabilities to analyze rule similarity, evaluate threat coverage and performance metrics, and deliver optimized recommendations for refining the rule set. By automating the rule optimization process, RuleGenie allows security teams to focus on more strategic tasks while enhancing the efficiency of SIEM systems and strengthening organizations' security posture. We evaluated RuleGenie on a comprehensive set of real-world SIEM rule formats, including Splunk, Sigma, and AQL (Ariel query language), demonstrating its platform-agnostic capabilities and adaptability across diverse security infrastructures. Our experimental results show that RuleGenie can effectively identify redundant rules, which in turn decreases false positive rates and enhances overall rule efficiency.

Figures (3)

• Figure 1: An overview of RuleGenie's three-phase pipeline.
• Figure 2: An example of a randomly selected rule. Sigma rule "New Service Uses Double Ampersand in Path" embedding using transformer models.
• Figure 3: Precision-recall curve for $k$ in Sigma rule set.