Living Off the LLM: How LLMs Will Change Adversary Tactics

TL;DR

This paper examines how on-device LLMs could empower adversaries to perform Living Off the LLM (LOLLM) attacks, expanding traditional LOTL capabilities through autonomous, multi-stage code generation and tool use. It offers a preliminary PoC demonstrating how a target’s locally deployed LLMs, with varying safety alignments, can be hijacked via jailbreaking approaches and scripted workflows to achieve persistence or data manipulation, all offline. To counter these threats, the authors advocate defense-in-depth mechanisms such as prompt firewalls, output sanitization, IOA-based anomaly detection, restricted tool usage, and crowdsourced rules to identify and mitigate LLM abuse. The work highlights the practical risks of on-device LLMs for security, emphasizes model-alignment as a critical attack surface, and provides concrete directions for detection and mitigation that can inform defenders and policy makers.”

Abstract

In living off the land attacks, malicious actors use legitimate tools and processes already present on a system to avoid detection. In this paper, we explore how the on-device LLMs of the future will become a security concern as threat actors integrate LLMs into their living off the land attack pipeline and ways the security community may mitigate this threat.