Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Plug-and-Play Method for Improving Imperceptibility and Capacity in Practical Generative Text Steganography

Kaiyi Pang

TL;DR

This work tackles the practical imperceptibility and capacity limitations of generative text steganography by introducing FreStega, a plug-and-play method that reconstructs language-model distributions at decoding time through sequential entropy-based adjustments and spatial target-domain guidance. By aligning the stego-text distribution with the target cover domain and mitigating distribution sharpening, FreStega enhances imperceptibility (higher MAUVE, lower detector F1) and embedding capacity (up to ~21% gains) without sacrificing linguistic quality. The approach is model-agnostic, compatible with prompt-based and fine-tuning methods, and effective across multiple LLM sizes and domains, requiring only modest target-domain data for strong domain alignment. Experimental results on IMDB, Shakespeare, and XHS across four models and several baselines demonstrate robust improvements in real-world scenarios with practical time efficiency and broad applicability.

Abstract

Linguistic steganography embeds secret information into seemingly innocuous text to safeguard privacy under surveillance. Generative linguistic steganography leverages the probability distributions of language models (LMs) and applies steganographic algorithms during generation, and has attracted increasing attention with the rise of large language models (LLMs). To strengthen security, prior work has focused on distribution-preserving steganographic algorithms that minimize the gap between stego sampling and random sampling from the model. However, their reliance on model distributions, which often deviate from real-world cover texts, leads to limited imperceptibility when facing steganalysis detectors in practical settings. Moreover, LLM distributions tend to be more deterministic, reducing entropy and thus lowering embedding capacity. In this paper, we propose a plug-and-play method that reconstructs the distributions of language models used for generative linguistic steganography. FreStega dynamically adjusts token probabilities from the language model at each step of autoregressive stego text generation, leveraging both sequential and spatial dimensions. Extensive experiments on four LLMs, three benchmark datasets, and four distribution-preserving steganographic baselines demonstrate that, by reforming the distribution, FreStega improves the imperceptibility of stego text in realistic scenarios and increases steganographic capacity by 15.41\%, without degrading the quality of the generated stegotext.

A Plug-and-Play Method for Improving Imperceptibility and Capacity in Practical Generative Text Steganography

TL;DR

This work tackles the practical imperceptibility and capacity limitations of generative text steganography by introducing FreStega, a plug-and-play method that reconstructs language-model distributions at decoding time through sequential entropy-based adjustments and spatial target-domain guidance. By aligning the stego-text distribution with the target cover domain and mitigating distribution sharpening, FreStega enhances imperceptibility (higher MAUVE, lower detector F1) and embedding capacity (up to ~21% gains) without sacrificing linguistic quality. The approach is model-agnostic, compatible with prompt-based and fine-tuning methods, and effective across multiple LLM sizes and domains, requiring only modest target-domain data for strong domain alignment. Experimental results on IMDB, Shakespeare, and XHS across four models and several baselines demonstrate robust improvements in real-world scenarios with practical time efficiency and broad applicability.

Abstract

Linguistic steganography embeds secret information into seemingly innocuous text to safeguard privacy under surveillance. Generative linguistic steganography leverages the probability distributions of language models (LMs) and applies steganographic algorithms during generation, and has attracted increasing attention with the rise of large language models (LLMs). To strengthen security, prior work has focused on distribution-preserving steganographic algorithms that minimize the gap between stego sampling and random sampling from the model. However, their reliance on model distributions, which often deviate from real-world cover texts, leads to limited imperceptibility when facing steganalysis detectors in practical settings. Moreover, LLM distributions tend to be more deterministic, reducing entropy and thus lowering embedding capacity. In this paper, we propose a plug-and-play method that reconstructs the distributions of language models used for generative linguistic steganography. FreStega dynamically adjusts token probabilities from the language model at each step of autoregressive stego text generation, leveraging both sequential and spatial dimensions. Extensive experiments on four LLMs, three benchmark datasets, and four distribution-preserving steganographic baselines demonstrate that, by reforming the distribution, FreStega improves the imperceptibility of stego text in realistic scenarios and increases steganographic capacity by 15.41\%, without degrading the quality of the generated stegotext.
Paper Structure (34 sections, 1 theorem, 17 equations, 13 figures, 14 tables, 1 algorithm)

This paper contains 34 sections, 1 theorem, 17 equations, 13 figures, 14 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Method
  3. Narrowing the Gap Toward Real-World Scenarios
  4. Overview
  5. Sequential Dimension
  6. Spatial Dimension
  7. Experiments
  8. Experiment Setup
  9. Baselines
  10. Datasets
  11. Models
  12. Metrics
  13. Linguistic Quality
  14. Capacity
  15. Domain Statistical Imperceptibility
  16. ...and 19 more sections

Key Result

Lemma 1

If $\delta \leq p, q \leq 1-\delta$ and $|p-q| \geq \delta$, then the following holds: By symmetry, the same inequality applies to $\left| \log\left(1 - \frac{q - p}{1 - p}\right) \right|$.

Figures (13)

  • Figure 1: Generative Linguistic Steganography Framework: Alice and Bob covertly communicate over a public channel monitored by Eve. Alice embeds secret information during the language model's token selection process to generate stego text. Using a shared key, language model, and prompt, Bob then recovers the secret information upon receiving the stego text.
  • Figure 2: Token frequency distribution comparison between text generated by state-of-the-art distribution-preserving algorithms (METEOR meteor2021, ADG zhang2021provably, DISCOP ding2023discop), Random Sampling (RS), and cover text on the IMDB IMDB using various LLMs (In each model scenario, the tokenizer corresponding to that specific model is used for tokenization and frequency statistics. Therefore, the distribution of the same human corpus may vary slightly after tokenization by different models).
  • Figure 3: F1 classification scores of state-of-the-art steganography methods (ADG zhang2021provably, METEOR meteor2021, DISCOP ding2023discop) against classic steganalysis classifiers (TS-CSW TS-CSW, TS-RNN TS-RNN, and R-BiLSTM-C r-bilstm-c) on the IMDB IMDB and SHAKESPEARE shakespere in real-world scenarios.
  • Figure 4: Overview of FreStega: We first adjust the temperature at each time step based on the instantaneous entropy in the sequential dimension. Then, we refine the language model’s distribution in the spatial dimension, aligning it with the target domain’s cover distribution.
  • Figure 5: Hyperparameter analysis of $\alpha$ ($c$=0.1). We tested the F1 scores of classifiers on three datasets (XHS, SHAKESPEARE, IMDB) using three classic steganalysis methods (TS-CSW TS-CSW, TS-RNN TS-RNN, TS-r-bilistm-c r-bilstm-c). The corresponding colored horizontal lines indicate the F1 scores of the steganalysis methods without FreStega. The green dashed line represents the MAUVE score relative to the target domain text.
  • ...and 8 more figures

Theorems & Definitions (2)

  • Lemma 1
  • proof