Unelicitable Backdoors in Language Models via Cryptographic Transformer Circuits

TL;DR

This work introduces unelicitable backdoors for language models by embedding cryptographic transformer circuits, realized through the Stravinsky language, to enable backdoors that cannot be triggered by polynomial-time elicitation methods. The authors present two concrete designs—NP-Complete and encrypted backdoors—and a password-locked baseline, plus a hardness scale and model-organism benchmarks to study elicitation difficulty. Empirical results show that the encrypted backdoor resists latent adversarial training and input-space attacks, while NP-Complete backdoors resist some elicitation but can be breached by certain activation-based tactics; password-locked backdoors remain vulnerable to standard red-teaming. The work highlights critical security gaps in pre-deployment mitigation for open-weight models and motivates developing stronger defenses, probing the offence-defence balance in AI safety, including potential cryptographic or information-theoretic safeguards.

Abstract

The rapid proliferation of open-source language models significantly increases the risks of downstream backdoor attacks. These backdoors can introduce dangerous behaviours during model deployment and can evade detection by conventional cybersecurity monitoring systems. In this paper, we introduce a novel class of backdoors in transformer models, that, in contrast to prior art, are unelicitable in nature. Unelicitability prevents the defender from triggering the backdoor, making it impossible to properly evaluate ahead of deployment even if given full white-box access and using automated techniques, such as red-teaming or certain formal verification methods. We show that our novel construction is not only unelicitable thanks to using cryptographic techniques, but also has favourable robustness properties. We confirm these properties in empirical investigations, and provide evidence that our backdoors can withstand state-of-the-art mitigation strategies. Additionally, we expand on previous work by showing that our universal backdoors, while not completely undetectable in white-box settings, can be harder to detect than some existing designs. By demonstrating the feasibility of seamlessly integrating backdoors into transformer models, this paper fundamentally questions the efficacy of pre-deployment detection strategies. This offers new insights into the offence-defence balance in AI safety and security.

Figures (7)

• Figure 1: Our encrypted LM backdoor design on the right, in comparison to an image classifier backdoor on the left. Inside any existing transformer-based language model, we insert an encrypted payload which affects the model outputs only in the presence of a certain trigger, and is entirely unelicitable otherwise.
• Figure 2: Our backdoor designs. a) The NP-Complete backdoor triggers only when the input encodes a solution to 3-SAT instance hardcoded into the model. b) The encrypted backdoor triggers only when provided with the key to a digital locker hardcoded into the model. For both designs the trigger is infeasible to find. The second design also makes the backdoor behaviour computationally infeasible to determine.
• Figure 3: The number of adversarial training steps necessary to elicit the backdoor in our password-locked, NP-hard, and encrypted models. Latent adversarial training successfully elicits the backdoor in the password-locked and NP-hard models, but fails to do so in $10^4$ steps in the encrypted case.
• Figure 4: LAT on the first layer eliciting the backdoors in the noisy password-locked and NP-complete constructions, while failing to elicit the noisy encrypted backdoor. The loss on the y-axis represents the fraction of the total possible loss that LAT has attained; a loss of 1.0 means the backdoor has been successfully elicited. The dotted lines show each of 34 separate runs, and the thick lines represent the average with 95% confidence intervals.
• Figure 5: A scale comparing various backdoor attacks (red team) and mitigation strategies (blue team) that are based on eliciting backdoor behaviours. For example, NP-complete verification backdoor cannot be elicited by GCG, but it is defeated by Latent Adversarial Training.