Semantic Gravity Wells: Why Negative Constraints Backfire

TL;DR

The paper investigates why large language models fail negative constraints, showing that violations follow a predictable logistic relationship with semantic pressure: $p(\text{violation}) = \sigma(-2.40 + 2.27\cdot P_0)$. It identifies a suppression asymmetry where constraints reduce target probability by $22.8$ percentage points in successes but only by $5.2$ points in failures, a $4.4\times$ difference. Mechanistic analyses reveal two failure modes: priming failures, where naming the forbidden word activates the target, and override failures, where late-layer feed-forward contributions overwhelm suppression; activations in layers 23–27 causally drive violations. Practically, the work suggests avoiding explicit naming of forbidden words and employing post-generation filtering for high-pressure cases, contributing to mechanistic interpretability by linking behavior to specific internal processes.

Abstract

Negative constraints (instructions of the form "do not use word X") represent a fundamental test of instruction-following capability in large language models. Despite their apparent simplicity, these constraints fail with striking regularity, and the conditions governing failure have remained poorly understood. This paper presents the first comprehensive mechanistic investigation of negative instruction failure. We introduce semantic pressure, a quantitative measure of the model's intrinsic probability of generating the forbidden token, and demonstrate that violation probability follows a tight logistic relationship with pressure ($p=σ(-2.40+2.27\cdot P_0)$; $n=40{,}000$ samples; bootstrap $95%$ CI for slope: $[2.21,,2.33]$). Through layer-wise analysis using the logit lens technique, we establish that the suppression signal induced by negative instructions is present but systematically weaker in failures: the instruction reduces target probability by only 5.2 percentage points in failures versus 22.8 points in successes -- a $4.4\times$ asymmetry. We trace this asymmetry to two mechanistically distinct failure modes. In priming failure (87.5% of violations), the instruction's explicit mention of the forbidden word paradoxically activates rather than suppresses the target representation. In override failure (12.5%), late-layer feed-forward networks generate contributions of $+0.39$ toward the target probability -- nearly $4\times$ larger than in successes -- overwhelming earlier suppression signals. Activation patching confirms that layers 23--27 are causally responsible: replacing these layers' activations flips the sign of constraint effects. These findings reveal a fundamental tension in negative constraint design: the very act of naming a forbidden word primes the model to produce it.

Figures (7)

• Figure 1: Violation rate increases monotonically with semantic pressure. Each point shows the observed violation rate within a pressure bin (horizontal bars: bin width; vertical bars: bootstrap 95% CI). The gray curve shows the logistic fit. The relationship is striking in its regularity: at $P_0 = 0.1$, only 9% of samples violate; at $P_0 = 0.9$, violations exceed 46%. The logistic model explains 78% of variance ($R^2 = 0.78$).
• Figure 2: Suppression is real but $4.4\times$ weaker in failures. Mean suppression magnitude $\Delta P = P_0 - P_1$ at the decision step. Both successes and failures show positive $\Delta P$, confirming the instruction has some effect. However, the magnitude differs dramatically: 22.8 percentage points in successes versus only 5.2 points in failures. Error bars show bootstrap 95% CI; intervals are non-overlapping.
• Figure 3: Failures attend to the target mention more than the negation. Attention metrics by pressure bin and outcome. Left: Instruction Attention Ratio. Center: Negation Focus. Right: Target-Mention Focus. At high pressure ($P_0 > 0.8$), failures show elevated TMF (0.205 vs. 0.200) and reduced NF (0.141 vs. 0.142). The pattern suggests that failures process the instruction's mention of X as a cue rather than a prohibition.
• Figure 4: Target probability emerges dramatically in late layers, and failures diverge from successes. Logit lens probabilities by layer for baseline (solid) and negative instruction (dashed) conditions, stratified by outcome (blue: success; red: failure). Three regimes are visible: (1) Early layers (0--20): all conditions show $P(X) < 10^{-4}$, no differentiation. (2) Critical layers (21--27): explosive divergence; failures surge while successes remain suppressed. (3) Final layer: baseline/failure reaches 0.71; negative/success stays at 0.08.
• Figure 5: Attention suppresses; FFN promotes---and FFN wins in failures. Layer-wise decomposition of contributions to target probability (layers 18--27). Attention contributions (green) are frequently negative, indicating suppression. FFN contributions (orange) are consistently positive. Critical finding: at layer 27, FFN contribution in failures (+0.39) is nearly $4\times$ larger than in successes (+0.10). The FFN override overwhelms attention's suppression signal.

Theorems & Definitions (1)

• Definition 1: Semantic Pressure