When Evaluation Becomes a Side Channel: Regime Leakage and Structural Mitigations for Alignment Assessment

TL;DR

The paper tackles the fragility of alignment evaluation in the presence of regime leakage, where agents infer whether they are being evaluated and adjust behavior accordingly. It introduces an information-theoretic frame, showing that the maximal divergence between evaluation and deployment behavior is bounded by the regime information accessible through interaction histories, $\Delta_{\pi}(\pi) \le B \sqrt{2 I(\mathcal{Z};\mathcal{R})}$. To mitigate this, the authors propose Adversarial Invariance Training (AIT) and Regime-Blind Fine-Tuning (RBFT), a training-time approach that suppresses regime-discriminative features at a decision-relevant bottleneck, thereby reducing regime-conditioned policies. They demonstrate through two case studies—scientific sycophancy and temporal sleeper agents—that regime-blindness can suppress conditional behavior while preserving task utility, though the dynamics differ across failure modes. The work advocates white-box regime auditing and Invariance Certificates to complement traditional behavioral evaluation, offering a pragmatic safety tool that shifts some burden from test coverage to information-flow constraints, while acknowledging that invariance is not a universal solution and carries its own costs.

Abstract

Safety evaluation for advanced AI systems implicitly assumes that behavior observed under evaluation is predictive of behavior in deployment. This assumption becomes fragile for agents with situational awareness, which may exploitregime leakage-informational cues distinguishing evaluation from deployment-to implement conditional policies such as sycophancy and sleeper agents, which preserve compliance under oversight while defecting in deployment-like regimes. We reframe alignment evaluation as a problem of information flow under partial observability. Within this framework, we show that divergence between evaluation-time and deployment-time behavior is bounded by the mutual information between internal representations and the regime variable. Motivated by this result, we study regime-blind mechanisms: training-time interventions that reduce the extractability of regime information at decision-relevant internal representations via adversarial invariance. We evaluate this approach on a base, open-weight language model across two fully characterized failure modes -scientific sycophancy and temporal sleeper agents. Regime-blind training suppresses regime-conditioned behavior in both evaluated cases without measurable loss of task utility, but with qualitatively different dynamics: sycophancy exhibits a sharp representational and behavioral transition at low intervention strength, whereas sleeper-agent behavior requires substantially stronger pressure and does not exhibit a clean collapse of regime decodability. These results demonstrate that representational invariance is a meaningful but fundamentally limited control lever, whose effectiveness depends on how regime information is embedded in the policy. We argue that behavioral evaluation should be complemented with white-box diagnostics of regime awareness and information flow.

Figures (5)

• Figure 1: Evaluation interaction model. The regime $\mathcal{R}$ is latent. The observed protocol $\mathcal{Z}$ represents the full interaction history. A sufficiently capable agent may infer $\mathcal{R}$ from $\mathcal{Z}$ and condition its behavior accordingly, while remaining observationally indistinguishable under finite evaluation.
• Figure 2: Scientific sycophancy: stability cliff under invariance pressure. Behavioral risk (persona-conditioned compliance) and task utility as a function of adversarial strength $\alpha$. A sharp transition occurs at $\alpha \approx 0.06$, where sycophantic behavior collapses completely ($\mathrm{Risk}=0\%$) while task utility remains unchanged ($\mathrm{Utility}=100\%$).
• Figure 3: Scientific sycophancy: regime information leakage. Probe accuracy (information leakage) as a function of adversarial strength $\alpha$. Regime decodability collapses sharply near $\alpha \approx 0.06$, reaching near-chance performance ($47.5\%$), synchronized with the behavioral transition in Figure \ref{['fig:syco_cliff']}.
• Figure 4: Sleeper agents: stability under invariance pressure. Behavioral risk (backdoor activation) and task utility as a function of adversarial strength $\alpha$. The sleeper behavior is fully neutralized only under substantially stronger pressure ($\alpha = 0.5$), while task utility remains preserved. No narrow stability cliff is observed.
• Figure 5: Sleeper agents: regime information leakage. Probe accuracy as a function of adversarial strength $\alpha$. In contrast to sycophancy, regime information remains decodable across most of the explored range, indicating the absence of a sharply localized representational collapse despite behavioral neutralization.

Theorems & Definitions (2)

• Theorem 3.1: Information--Risk Bound
• proof