Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Building a Robust Risk-Based Access Control System to Combat Ransomware's Capability to Encrypt: A Machine Learning Approach

Kenan Begovic, Abdulaziz Al-Ali, Qutaibah Malluhi

TL;DR

The paper addresses ransomware-driven unauthorized encryption by proposing a risk-based access-control framework that couples real-time machine learning with SELinux policy enforcement on Linux. It leverages function-level tracing via Linux ftrace/function_graph to construct a high-resolution feature set (36 features, later unified for rules and ML) and deploys a dual-layer detector: a lightweight rule layer (ERULE) and a model layer (EML) based on XGBoost, both feeding into SELinux booleans rule_block and ml_block to gate write/append operations in encryption contexts. The methodology includes end-to-end dataset construction across kernel versions, graph-based feature extraction, contextual encoding, outlier handling, and careful partitioning, with explicit defender-centric utility metrics guiding feature selection. Experimental results show ms-scale end-to-end latency with strong detection quality, where the rule layer provides fast early blocks and the model layer offers depth and generalization, achieving a balanced trade-off between latency, overhead, and policy coverage. This work demonstrates a practical, explainable path to enforceable, risk-proportionate encryption control on production Linux systems, with clear guidance for reducing overhead and expanding deployment in enterprise environments.

Abstract

Ransomware core capability, unauthorized encryption, demands controls that identify and block malicious cryptographic activity without disrupting legitimate use. We present a probabilistic, risk-based access control architecture that couples machine learning inference with mandatory access control to regulate encryption on Linux in real time. The system builds a specialized dataset from the native ftrace framework using the function_graph tracer, yielding high-resolution kernel-function execution traces augmented with resource and I/O counters. These traces support both a supervised classifier and interpretable rules that drive an SELinux policy via lightweight booleans, enabling context-sensitive permit/deny decisions at the moment encryption begins. Compared to approaches centered on sandboxing, hypervisor introspection, or coarse system-call telemetry, the function-level tracing we adopt provides finer behavioral granularity than syscall-only telemetry while avoiding the virtualization/VMI overhead of sandbox-based approaches. Our current user-space prototype has a non-trivial footprint under burst I/O; we quantify it and recognize that a production kernel-space solution should aim to address this. We detail dataset construction, model training and rule extraction, and the run-time integration that gates file writes for suspect encryption while preserving benign cryptographic workflows. During evaluation, the two-layer composition retains model-level detection quality while delivering rule-like responsiveness; we also quantify operational footprint and outline engineering steps to reduce CPU and memory overhead for enterprise deployment. The result is a practical path from behavioral tracing and learning to enforceable, explainable, and risk-proportionate encryption control on production Linux systems.

Building a Robust Risk-Based Access Control System to Combat Ransomware's Capability to Encrypt: A Machine Learning Approach

TL;DR

The paper addresses ransomware-driven unauthorized encryption by proposing a risk-based access-control framework that couples real-time machine learning with SELinux policy enforcement on Linux. It leverages function-level tracing via Linux ftrace/function_graph to construct a high-resolution feature set (36 features, later unified for rules and ML) and deploys a dual-layer detector: a lightweight rule layer (ERULE) and a model layer (EML) based on XGBoost, both feeding into SELinux booleans rule_block and ml_block to gate write/append operations in encryption contexts. The methodology includes end-to-end dataset construction across kernel versions, graph-based feature extraction, contextual encoding, outlier handling, and careful partitioning, with explicit defender-centric utility metrics guiding feature selection. Experimental results show ms-scale end-to-end latency with strong detection quality, where the rule layer provides fast early blocks and the model layer offers depth and generalization, achieving a balanced trade-off between latency, overhead, and policy coverage. This work demonstrates a practical, explainable path to enforceable, risk-proportionate encryption control on production Linux systems, with clear guidance for reducing overhead and expanding deployment in enterprise environments.

Abstract

Ransomware core capability, unauthorized encryption, demands controls that identify and block malicious cryptographic activity without disrupting legitimate use. We present a probabilistic, risk-based access control architecture that couples machine learning inference with mandatory access control to regulate encryption on Linux in real time. The system builds a specialized dataset from the native ftrace framework using the function_graph tracer, yielding high-resolution kernel-function execution traces augmented with resource and I/O counters. These traces support both a supervised classifier and interpretable rules that drive an SELinux policy via lightweight booleans, enabling context-sensitive permit/deny decisions at the moment encryption begins. Compared to approaches centered on sandboxing, hypervisor introspection, or coarse system-call telemetry, the function-level tracing we adopt provides finer behavioral granularity than syscall-only telemetry while avoiding the virtualization/VMI overhead of sandbox-based approaches. Our current user-space prototype has a non-trivial footprint under burst I/O; we quantify it and recognize that a production kernel-space solution should aim to address this. We detail dataset construction, model training and rule extraction, and the run-time integration that gates file writes for suspect encryption while preserving benign cryptographic workflows. During evaluation, the two-layer composition retains model-level detection quality while delivering rule-like responsiveness; we also quantify operational footprint and outline engineering steps to reduce CPU and memory overhead for enterprise deployment. The result is a practical path from behavioral tracing and learning to enforceable, explainable, and risk-proportionate encryption control on production Linux systems.
Paper Structure (57 sections, 3 equations, 6 figures, 16 tables)

This paper contains 57 sections, 3 equations, 6 figures, 16 tables.

Table of Contents

  1. Introduction
  2. Threat model and enforcement boundary.
  3. Related Work
  4. Access Control Models and Risk-Based Approaches
  5. Machine Learning with Kernel-Level Tracing for Encryption Detection
  6. eBPF-Based Approaches
  7. Kernel-Level System Call Tracing
  8. Low-Level Memory and Storage Monitoring
  9. Datasets for Ransomware Detection
  10. Methodology
  11. Design Rationale and Objectives
  12. Research Questions
  13. Dataset Construction and Collection Methodology
  14. Dataset Preprocessing and Cleaning Methodology
  15. Graph-Based Feature Extraction from Function Traces
  16. ...and 42 more sections

Figures (6)

  • Figure 1: Compact hypothesis view: early behavioral evidence supports probabilistic detection, and a context-weighted risk policy drives OS-native enforcement through SELinux.
  • Figure 2: Permutation‑importance elbow for the rule detector ($k^{\star}=2$).
  • Figure 3: χ2+utility elbow for the ML detector ($k^{\star}=36$).
  • Figure 4: CIL sketch for risk-based gating in SELinux. The booleanif is the choke point governed by $E\textsubscript{RULE}$/$E\textsubscript{ML}$ detectors.
  • Figure 5: Dual-layer enforcement prototype
  • ...and 1 more figures