Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Post-Training Local LLM Agents for Linux Privilege Escalation with Verifiable Rewards

Philipp Normann, Andreas Happe, Jürgen Cito, Daniel Arp

Abstract

LLM agents are increasingly relevant to research domains such as vulnerability discovery. Yet, the strongest systems remain closed and cloud-only, making them resource-intensive, difficult to reproduce, and unsuitable for work involving proprietary code or sensitive data. Consequently, there is an urgent need for small, local models that can perform security tasks under strict resource budgets, but methods for developing them remain underexplored. In this paper, we address this gap by proposing a two-stage post-training pipeline. We focus on the problem of Linux privilege escalation, where success is automatically verifiable and the task requires multi-step interactive reasoning. Using an experimental setup that prevents data leakage, we post-train a 4B model in two stages: supervised fine-tuning on traces from procedurally generated privilege-escalation environments, followed by reinforcement learning with verifiable rewards. On a held-out benchmark of 12 Linux privilege-escalation scenarios, supervised fine-tuning alone more than doubles the baseline success rate at 20 rounds, and reinforcement learning further lifts our resulting model, PrivEsc-LLM, to 95.8%, nearly matching Claude Opus 4.6 at 97.5%. At the same time, the expected inference cost per successful escalation is reduced by over 100x.

Post-Training Local LLM Agents for Linux Privilege Escalation with Verifiable Rewards

Abstract

LLM agents are increasingly relevant to research domains such as vulnerability discovery. Yet, the strongest systems remain closed and cloud-only, making them resource-intensive, difficult to reproduce, and unsuitable for work involving proprietary code or sensitive data. Consequently, there is an urgent need for small, local models that can perform security tasks under strict resource budgets, but methods for developing them remain underexplored. In this paper, we address this gap by proposing a two-stage post-training pipeline. We focus on the problem of Linux privilege escalation, where success is automatically verifiable and the task requires multi-step interactive reasoning. Using an experimental setup that prevents data leakage, we post-train a 4B model in two stages: supervised fine-tuning on traces from procedurally generated privilege-escalation environments, followed by reinforcement learning with verifiable rewards. On a held-out benchmark of 12 Linux privilege-escalation scenarios, supervised fine-tuning alone more than doubles the baseline success rate at 20 rounds, and reinforcement learning further lifts our resulting model, PrivEsc-LLM, to 95.8%, nearly matching Claude Opus 4.6 at 97.5%. At the same time, the expected inference cost per successful escalation is reduced by over 100x.
Paper Structure (17 sections, 4 equations, 3 figures, 4 tables)

This paper contains 17 sections, 4 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Problem Setting and Agent Interface
  5. Procedural Environment Generation and Leakage Controls
  6. Trace Collection and SFT Dataset Construction
  7. Two-Stage Post-Training
  8. Evaluation Protocol and Reproducibility
  9. Experimental Setup
  10. Results
  11. Main Results
  12. Per-Scenario Analysis
  13. Training and Inference Cost
  14. Failure Analysis
  15. Discussion
  16. ...and 2 more sections

Figures (3)

  • Figure 1: $P(\text{root} \mid R)$ across interaction budgets ($R \in \{5,10,\ldots,60\}$). Each curve reports the fraction of runs that achieve root within $R$ rounds; shaded bands show 95% Wilson CIs. Sample size is $N{=}120$ runs per model.
  • Figure 2: Per-scenario success at the 60-round budget, where each cell shows $x/10$ successful runs and color encodes the corresponding success rate.
  • Figure 3: Per-run success at $R{=}20$ versus expected cost per successful root at the same budget (log-scale x-axis). Cost is expected per-run cost divided by $P(\text{root} \mid R{=}20)$, so the desirable region is upper-left. Our two-stage pipeline moves Qwen3-4B closer to the frontier, with gains from SFT and RL.