Two is Better than One: Efficient Ensemble Defense for Robust and Compact Models

TL;DR

This work addresses the challenge of deploying robust yet compact CNNs on resource-constrained devices by introducing Efficient Ensemble Defense (EED), which constructs a diverse ensemble from a single base model through multiple pruning importance scores and data subsetting. EED leverages three mechanisms—robust diversity, misclassification-focused regularization, and compactness regularization—together with Dynamic Inference Ensemble to add sub-models only as needed during inference, achieving both high adversarial robustness and significant speedups. Experimental results on CIFAR-10 and SVHN show state-of-the-art robustness against PGD, AutoAttack, C&W, and DeepFool for compressed models, with up to 1.86x speedups and consistent performance across ResNet-18 and VGG-16. The approach demonstrates practical applicability for edge and mobile deployments, offering a scalable path to robust, compact models without training an ensemble of separate full-size networks.

Abstract

Deep learning-based computer vision systems adopt complex and large architectures to improve performance, yet they face challenges in deployment on resource-constrained mobile and edge devices. To address this issue, model compression techniques such as pruning, quantization, and matrix factorization have been proposed; however, these compressed models are often highly vulnerable to adversarial attacks. We introduce the \textbf{Efficient Ensemble Defense (EED)} technique, which diversifies the compression of a single base model based on different pruning importance scores and enhances ensemble diversity to achieve high adversarial robustness and resource efficiency. EED dynamically determines the number of necessary sub-models during the inference stage, minimizing unnecessary computations while maintaining high robustness. On the CIFAR-10 and SVHN datasets, EED demonstrated state-of-the-art robustness performance compared to existing adversarial pruning techniques, along with an inference speed improvement of up to 1.86 times. This proves that EED is a powerful defense solution in resource-constrained environments.

Figures (6)

• Figure 1: Challenges in AP and the potential of ensemble methods: AP weakens the network by removing critical parameters, reducing robustness and increasing vulnerability to adversarial attacks. An ensemble approach mitigates pruning effects by combining pruned models, enhancing robustness through their collective strength (illustrated in green).
• Figure 2: Importance score visualization for the 4-5-6-3 CNN model on the MNIST dataset: yellow/red indicate higher importance, green/blue indicate lower.
• Figure 3: Robust performance by $s_r$.
• Figure 4: Correlation between RD and robustness in different ensemble components.
• Figure 5: Parameter distribution in each layer of VGG16 pruned to 90% sparsity with AT.