Position: Towards Resilience Against Adversarial Examples

TL;DR

The paper argues that robustness against a single or known set of attacks is insufficient for real-world deployments, as attackers can uncover novel perturbations post-deployment. It introduces adversarial resilience and a simplified continual adaptive robustness (CAR) framework wherein defenses can update quickly in response to newly discovered attacks, guided by time-evolving attacker and defender knowledge sets. CAR bridges to existing concepts like simultaneous multiattack robustness (sMAR) and unforeseen attack robustness (UAR), indicating that advances in those areas can directly enhance resilience. The work also outlines practical applications, challenges such as label scarcity and poisoning risks, and open directions for research, including standardized evaluation and efficient fine-tuning strategies for rapid adaptation. Overall, the authors advocate for a shift from static robustness to adaptive resilience to ensure sustained protection against evolving adversarial threats.

Abstract

Current research on defending against adversarial examples focuses primarily on achieving robustness against a single attack type such as $\ell_2$ or $\ell_{\infty}$-bounded attacks. However, the space of possible perturbations is much larger than considered by many existing defenses and is difficult to mathematically model, so the attacker can easily bypass the defense by using a type of attack that is not covered by the defense. In this position paper, we argue that in addition to robustness, we should also aim to develop defense algorithms that are adversarially resilient -- defense algorithms should specify a means to quickly adapt the defended model to be robust against new attacks. We provide a definition of adversarial resilience and outline considerations of designing an adversarially resilient defense. We then introduce a subproblem of adversarial resilience which we call continual adaptive robustness, in which the defender gains knowledge of the formulation of possible perturbation spaces over time and can then update their model based on this information. Additionally, we demonstrate the connection between continual adaptive robustness and previously studied problems of multiattack robustness and unforeseen attack robustness and outline open directions within these fields which can contribute to improving continual adaptive robustness and adversarial resilience.

Figures (1)

• Figure 2: Full knowledge vs knowledge mismatch. The white box represents the space of possible perturbations that we would expect a model to be robust to (ie. space of imperceptible perturbations) which we may not know how to model. The green oval represents the space of perturbations captured in the defender knowledge set and the red oval represents the space of perturbations captured in the attacker knowledge set. (A) Full knowledge occurs when the defender knows the space known to the attacker while (B) Knowledge mismatch occurs when there exist perturbations known to the attacker that are not known to the defender. Robustness in this setting corresponds to unforeseen attack robustness.

Theorems & Definitions (6)

• Definition 2.1: Single Attack Game
• Definition 3.1: Adaptive Attacker Knowledge Set
• Definition 3.2: Adaptive Defense Algorithm
• Definition 3.3: Adversarial Resilience
• Definition 6.1: Adaptive Defender Knowledge Set
• Definition 6.2: Continual Adaptive Robustness