Hide and Seek in Embedding Space: Geometry-based Steganography and Detection in Large Language Models
Charles Westphal, Keivan Navaie, Fernando E. Rosas
TL;DR
The paper investigates covert data exfiltration through fine-tuned LLMs and formalizes payload recoverability via two axes, $ R_{bucket}$ and $ R_{secret}$, to distinguish recoverable versus unrecoverable leakage. It introduces low-recoverability steganography based on embedding-space hyperplane projections to encode secrets and bucket tokens, significantly reducing recoverability across models (e.g., improvements in exact secret recovery from 9–24% to 19–43% depending on model) while preserving fluency. Because traditional distributional steganalysis is ill-suited for fine-tuning attacks, the authors develop a mechanistic detection approach using linear probes on late-layer activations, uncovering internal signatures that indicate steganographic encoding with up to a 33% gain in detection accuracy over base models. They also show that a secret-conditioned XOR masking control can obfuscate output-level recoverability, while internal probes still reveal traces of steganographic fine-tuning, highlighting the potential for internal-state–based defenses. The work advances both offensive techniques and defensive mechanisms for detecting malicious fine-tuning in deployed LLMs, with practical implications for model providers and secure deployments.
Abstract
Fine-tuned LLMs can covertly encode prompt secrets into outputs via steganographic channels. Prior work demonstrated this threat but relied on trivially recoverable encodings. We formalize payload recoverability via classifier accuracy and show previous schemes achieve 100\% recoverability. In response, we introduce low-recoverability steganography, replacing arbitrary mappings with embedding-space-derived ones. For Llama-8B (LoRA) and Ministral-8B (LoRA) trained on TrojanStego prompts, exact secret recovery rises from 17$\rightarrow$30\% (+78\%) and 24$\rightarrow$43\% (+80\%) respectively, while on Llama-70B (LoRA) trained on Wiki prompts, it climbs from 9$\rightarrow$19\% (+123\%), all while reducing payload recoverability. We then discuss detection. We argue that detecting fine-tuning-based steganographic attacks requires approaches beyond traditional steganalysis. Standard approaches measure distributional shift, which is an expected side-effect of fine-tuning. Instead, we propose a mechanistic interpretability approach: linear probes trained on later-layer activations detect the secret with up to 33\% higher accuracy in fine-tuned models compared to base models, even for low-recoverability schemes. This suggests that malicious fine-tuning leaves actionable internal signatures amenable to interpretability-based defenses.