SHIELD: Secure Hypernetworks for Incremental Expansion Learning Defense

TL;DR

We address the challenge of robust continual learning by proposing SHIELD, a framework that unifies certifiable adversarial robustness with sequential task adaptation. SHIELD uses a hypernetwork to generate task-specific target models from compact embeddings and employs Interval Bound Propagation to provide formal robustness guarantees on interval inputs, complemented by Interval MixUp to tighten bounds and smooth decision boundaries. The core contributions are the SHIELD architecture, the Interval MixUp training strategy, and theoretical robustness guarantees with empirical validation across diverse benchmarks, including CIL scenarios and TinyImageNet. SHIELD demonstrates state-of-the-art or competitive robust performance while maintaining scalability and privacy by avoiding replay buffers and full model copies, marking a significant step toward practical robust lifelong learning under adversarial threats.

Abstract

Continual learning under adversarial conditions remains an open problem, as existing methods often compromise either robustness, scalability, or both. We propose a novel framework that integrates Interval Bound Propagation (IBP) with a hypernetwork-based architecture to enable certifiably robust continual learning across sequential tasks. Our method, SHIELD, generates task-specific model parameters via a shared hypernetwork conditioned solely on compact task embeddings, eliminating the need for replay buffers or full model copies and enabling efficient over time. To further enhance robustness, we introduce Interval MixUp, a novel training strategy that blends virtual examples represented as $\ell_{\infty}$ balls centered around MixUp points. Leveraging interval arithmetic, this technique guarantees certified robustness while mitigating the wrapping effect, resulting in smoother decision boundaries. We evaluate SHIELD under strong white-box adversarial attacks, including PGD and AutoAttack, across multiple benchmarks. It consistently outperforms existing robust continual learning methods, achieving state-of-the-art average accuracy while maintaining both scalability and certification. These results represent a significant step toward practical and theoretically grounded continual learning in adversarial settings.

Figures (11)

• Figure 1: SHIELD uses a hypernetwork to map task-specific embeddings $\boldsymbol{e}_t$ into target models that propagate virtual Interval MixUp hypercubes, enabling robust multi-task learning and certified adversarial resistance. In the Interval MixUp visualization, edge images are perturbed, the center is unperturbed, and the IBP column shows how the input hypercube is transformed across network layers.
• Figure 2: (left side) Without Interval MixUp: Training leads to sharper boundaries and poor robustness across class transitions. (right side) With Interval MixUp: Interpolated samples ($\times$) with scaled certified regions (boxes) encourage smooth transitions and robust boundaries.
• Figure 3: Comparison of verified accuracy and classical accuracy across four continual learning benchmarks: Permuted MNIST, Rotated MNIST, Split CIFAR-100, and Split miniImageNet. For each task, we report the AA metric after sequentially learning all preceding tasks.
• Figure 4: Comparison of different epsilon decay rates used in Interval MixUp.
• Figure 5: Comparison of results for different epsilon decay rates across (a) Permuted MNIST, (b) Split CIFAR-100, and (c) Split miniImageNet. Top row: AA measured immediately after learning each task. Bottom row: AA evaluated on all tasks after completing the final task.

Theorems & Definitions (5)

• Definition 3.1: Certified Robustness
• Definition 3.2: Certified Robustness in Continual Learning
• Theorem 3.1: Certified Robustness Preservation
• Definition 8.1: MixUp
• proof