Backdoor defense, learnability and obfuscation
Paul Christiano, Jacob Hilton, Victor Lecomte, Mark Xu
TL;DR
This work introduces a formal game-theoretic notion of defendability against backdoors, requiring runtime trigger detection under a random-trigger constraint that breaks symmetry between clean and backdoored functions. It links defendability to learnability: in the unbounded setting it aligns with VC dimension, while in the computational setting efficient defendability follows from efficient PAC learning but does not imply it, and can fail under obfuscation, as shown with polynomial-size circuits. The paper also demonstrates a natural, fast defense for polynomial-size decision trees that outpaces learning, and discusses mechanistic defenses, separations from PAC learnability, and AI-alignment implications. Overall, efficient defendability sits between efficient learnability and obfuscation, offering a nuanced lens on the trade-offs between detection capability, computational constraints, and the structure of the hypothesis class.
Abstract
We introduce a formal notion of defendability against backdoors using a game between an attacker and a defender. In this game, the attacker modifies a function to behave differently on a particular input known as the "trigger", while behaving the same almost everywhere else. The defender then attempts to detect the trigger at evaluation time. If the defender succeeds with high enough probability, then the function class is said to be defendable. The key constraint on the attacker that makes defense possible is that the attacker's strategy must work for a randomly-chosen trigger. Our definition is simple and does not explicitly mention learning, yet we demonstrate that it is closely connected to learnability. In the computationally unbounded setting, we use a voting algorithm of Hanneke et al. (2022) to show that defendability is essentially determined by the VC dimension of the function class, in much the same way as PAC learnability. In the computationally bounded setting, we use a similar argument to show that efficient PAC learnability implies efficient defendability, but not conversely. On the other hand, we use indistinguishability obfuscation to show that the class of polynomial size circuits is not efficiently defendable. Finally, we present polynomial size decision trees as a natural example for which defense is strictly easier than learning. Thus, we identify efficient defendability as a notable intermediate concept in between efficient learnability and obfuscation.