The Steganographic Potentials of Language Models

TL;DR

This work investigates the steganographic potential of LLMs and the risks it poses for alignment and oversight. It develops an encoder–decoder RL framework and conducts colors, direct steganography, and prompting experiments to characterize how payloads can be covertly embedded and decoded, with performance improvements up to $20\%$ for $3$-bit payloads and undetected accuracy reaching as high as $0.66$ in favorable prompting scenarios. The findings demonstrate that while current models exhibit rudimentary steganographic abilities, convergence is fragile and highly sensitive to prompts, model class, and training setup, underscoring the need for robust detection, mitigation, and normative frameworks for safe deployment. Overall, the paper highlights significant dual-use risks and the imperative to develop monitoring and defense mechanisms as steganographic capabilities scale with model power and deployment.

Abstract

The potential for large language models (LLMs) to hide messages within plain text (steganography) poses a challenge to detection and thwarting of unaligned AI agents, and undermines faithfulness of LLMs reasoning. We explore the steganographic capabilities of LLMs fine-tuned via reinforcement learning (RL) to: (1) develop covert encoding schemes, (2) engage in steganography when prompted, and (3) utilize steganography in realistic scenarios where hidden reasoning is likely, but not prompted. In these scenarios, we detect the intention of LLMs to hide their reasoning as well as their steganography performance. Our findings in the fine-tuning experiments as well as in behavioral non fine-tuning evaluations reveal that while current models exhibit rudimentary steganographic abilities in terms of security and capacity, explicit algorithmic guidance markedly enhances their capacity for information concealment.

Figures (6)

• Figure 1: Steganography encoding and learning scheme for colors experiment. Subfigure (\ref{['fig1:subim1']}) depicts an encoder, $f_\theta$, mapping a message $m$ (payload) from the message space $M$, into a cover $x$ in the cover space $C$. The decoder, $f_\theta^{-1}$, tries to map the cover $x$ back to the message, $\hat{m}$. (\ref{['fig1:subim2']}) Reinforcement learning (RL) loop where the encoder and decoder learn to generate new tokens with cover $x$ or message $m$ respectively (shown on the right), from queries (left) and receive rewards $R$ for accurate message decoding. The RL fine-tuning process employs gradient ascent to adjust the model weights, $\theta$, enhancing the encoder and decoder's ability to communicate the original message $m$ with higher fidelity.
• Figure 2: In the direct steganography experiment, Llama-3.1 8B Instruct learns to transmit a payload while summarizing text, under GPT 3.5 Turbo supervision for both task evaluation and steganography monitoring. The y-axis shows undetected accuracy (reward) on a held-out validation set before (baseline) and after fine-tuning ($n$ denotes sample size), while the x-axis displays the number of distinct payloads plus the case with no payload (e.g. if there are 8 payloads, then the cardinality is 9 and entropy is about 3 bits).
• Figure 3: The learned steganography scheme from the colors experiment for one fine-tuning run. Left: encoder's color-to-name mapping. Right: decoder's name-to-color mapping. Frequency is indicated by a color scale.
• Figure 4: Depiction of steganographic communication between two language model agents: an ENCODER and a DECODER. They engage in a conversation to establish shared COMMON KNOWLEDGE before the ENCODER is given a secret message ("DOLPHIN") after N turns. The ENCODER then embeds the secret within the conversation using this shared knowledge, while avoiding detection by an OVERSEER monitoring the exchange. The DECODER identifies hidden patterns to successfully retrieve the secret message, highlighting the model's ability to transmit information covertly under adversarial oversight.
• Figure 5: Illustration of an indirect steganography game scenario designed to evaluate the emergence of covert communication strategies in language models. The AI assistant is tasked with helping a Team guess a sequence of letters without directly revealing them, while an overseer monitors for rule violations. The AI must balance providing subtle hints to aid Team A, while avoiding detection to prevent penalties. The experimental setup pressures the AI to optimize hidden communication under adversarial oversight, as Team B is designed to always have an advantage.