Towards Automatic Hands-on-Keyboard Attack Detection Using LLMs in EDR Solutions
Amit Portnoy, Ehud Azikri, Shay Kels
TL;DR
The paper tackles Hands-on-Keyboard detection within EDR by transforming rich, heterogeneous endpoint activity into narrative 'endpoint stories' and applying fine-tuned LLMs to classify benign versus HOK-related events. It implements a windowed approach to manage long contexts, using either a BERT-small with a BiLSTM head or a Phi-2-based per-window embedding strategy, followed by a downstream classifier. Evaluation on production EDR data shows that LLM-based pipelines, especially when combining per-window embeddings with transformer-head classifiers, outperform a LightGBM baseline at a low false-positive rate, demonstrating practical potential for real-time security monitoring. The work contributes a data pipeline for narrative conversion, two viable LLM-based training regimes, and empirical evidence that LLMs can enhance HOK detection performance in operational EDR deployments.
Abstract
Endpoint Detection and Remediation (EDR) platforms are essential for identifying and responding to cyber threats. This study presents a novel approach using Large Language Models (LLMs) to detect Hands-on-Keyboard (HOK) cyberattacks. Our method involves converting endpoint activity data into narrative forms that LLMs can analyze to distinguish between normal operations and potential HOK attacks. We address the challenges of interpreting endpoint data by segmenting narratives into windows and employing a dual training strategy. The results demonstrate that LLM-based models have the potential to outperform traditional machine learning methods, offering a promising direction for enhancing EDR capabilities and apply LLMs in cybersecurity.