Subliminal Effects in Your Data: A General Mechanism via Log-Linearity

TL;DR

The paper presents Logit-Linear Selection (LLS), a general mechanism to induce subliminal changes in language models by filtering preference data with a log-linearity-based criterion and fine-tuning via Direct Preference Optimization. Grounded in a $\log$-linear representation of model probabilities, LLS argues that tiny per-example correlations accumulate to produce robust, architecture-wide shifts in behavior even without explicit prompts at inference. The authors provide theoretical justifications (including a formal correlation bound) and extensive experiments across animal preferences, translation, and misalignment personas, showing that the same dataset can transfer target traits across diverse teacher–student model pairs. They also discuss transfer dynamics, potential defenses, and practical applications such as dataset watermarking, highlighting both risks and opportunities for controlled, principled manipulation of model behavior. The work advances a principled framework for understanding how data shape downstream properties and suggests mechanisms to detect, defend against, and possibly harness subliminal learning in real-world settings.

Abstract

Training modern large language models (LLMs) has become a veritable smorgasbord of algorithms and datasets designed to elicit particular behaviors, making it critical to develop techniques to understand the effects of datasets on the model's properties. This is exacerbated by recent experiments that show datasets can transmit signals that are not directly observable from individual datapoints, posing a conceptual challenge for dataset-centric understandings of LLM training and suggesting a missing fundamental account of such phenomena. Towards understanding such effects, inspired by recent work on the linear structure of LLMs, we uncover a general mechanism through which hidden subtexts can arise in generic datasets. We introduce Logit-Linear-Selection (LLS), a method that prescribes how to select subsets of a generic preference dataset to elicit a wide range of hidden effects. We apply LLS to discover subsets of real-world datasets so that models trained on them exhibit behaviors ranging from having specific preferences, to responding to prompts in a different language not present in the dataset, to taking on a different persona. Crucially, the effect persists for the selected subset, across models with varying architectures, supporting its generality and universality.

Figures (19)

• Figure 1: System prompt transfer via Logit-Linear Selection (our algorithm) where the student model learns to respond in Spanish.
• Figure 2: Mean counts of animal mentions when ${\mathsf{M}}_{\mathsf{T}} = {\mathsf{M}}_{\mathsf{S}}$ are both Olmo2-7B-Instruct. For all examples the blue bars are essentially invisible as the base model ${\mathsf{M}}_{\mathsf{S}}$ (before fine-tuning) rarely mentions the animal without the system prompt. See \ref{['fig:animal-barchart-overflow']} in appendix for analogous plots for different student models.
• Figure 3: Training progress for animal: "owl" (\ref{['sec:experiments_animals']}).
• Figure 4: Generation from Olmo2-7B-Instruct student model. Teacher model (also Olmo2-7B-Instruct) was system prompted to love elephants and mention them frequently.
• Figure 5: Results for LLS with a system prompt instructing the model to answer in a different language. For all 3 plots, the teacher model ${\mathsf{M}}_{\mathsf{T}}$ is OLMo-2-7B, and the student model ${\mathsf{M}}_{\mathsf{S}}$ varies; see \ref{['sec:experiments_translator']}.

Theorems & Definitions (14)

• Definition 2.1: Linear Representations
• Theorem 2.2: Informal
• Theorem C.1
• Remark C.1
• Proposition C.3
• proof
• Definition C.2
• Lemma C.4
• proof
• Definition C.3: Embedding Vectors Form Well-Behaved Subspace