iCNN-LSTM: A batch-based incremental ransomware detection system using Sysmon

TL;DR

The paper introduces iCNN-LSTM, a batch-based incremental ransomware detector that processes Sysmon logs on Windows endpoints to enable real-time threat detection and continual adaptation without full retraining. The model combines parallel CNN and LSTM streams with an attention mechanism and updates its weights via mini-batches, addressing class imbalance with SMOTE. It achieves a remarkable $F_2$-score of 99.61% with 0.17% false positives and 4.69% false negatives on an imbalanced dataset spanning six ransomware families, outperforming existing incremental-learning approaches in both accuracy and runtime. The approach demonstrates strong practicality for real-time defense, while future work will address model degradation over time and resource usage. Overall, the framework advances real-time, adaptive ransomware detection by leveraging dynamic Sysmon streams and an efficient, hybrid CNN-LSTM architecture.

Abstract

In response to the increasing ransomware threat, this study presents a novel detection system that integrates Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks. By leveraging Sysmon logs, the system enables real-time analysis on Windows-based endpoints. Our approach overcomes the limitations of traditional models by employing batch-based incremental learning, allowing the system to continuously adapt to new ransomware variants without requiring complete retraining. The proposed model achieved an impressive average F2-score of 99.61\%, with low false positive and false negative rates of 0.17\% and 4.69\%, respectively, within a highly imbalanced dataset. This demonstrates exceptional accuracy in identifying malicious behaviour. The dynamic detection capabilities of Sysmon enhance the model's effectiveness by providing a reliable stream of security events, mitigating the vulnerabilities associated with static detection methods. Furthermore, the parallel processing of LSTM modules, combined with attention mechanisms, significantly improves training efficiency and reduces latency, making our system well-suited for real-world applications. These findings underscore the potential of our CNN-LSTM framework as a robust solution for real-time ransomware detection, ensuring adaptability and resilience in the face of evolving cyber threats.

Figures (11)

• Figure 1: The proposed batch-based incremental ransomware detection framework using Sysmon. Blue arrows indicate the data stream that trains the model, while red arrows represent the data flow to be classified. Additionally, the blue dashed line symbolises the logical step of baselining the model, distinguishing it from actual data flows.
• Figure 2: The laboratory setup used to collect both ransomware and goodware events to build the dataset.
• Figure 3: fastText conversion to vectors using n-grams. The above example shows the process of converting the word "explain" into vectors where $v_n$ represents the vector representation of the corresponding $n$-gram
• Figure 4: A heatmap showing the correlations between features within the training dataset
• Figure 5: The structure of an LSTM model. The red region denoted by $f_t$ outlines the forget gate; The blue shaded region denoted by $i_t$ represents the input gate, and the orange region ($O_t$) represents the output gate