Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Harmonizing Intra-coherence and Inter-divergence in Ensemble Attacks for Adversarial Transferability

Zhaoyang Ma, Zhihao Wu, Wang Lu, Xin Gao, Jinghang Yue, Taolin Zhang, Lipo Wang, Youfang Lin, Jing Wang

TL;DR

HEAT addresses the limited capture of shared gradient directions and the lack of adaptive weighting in ensemble adversarial attacks by injecting domain generalization into the attack design. It introduces Consensus Gradient Direction Synthesizer (C-GRADS) to extract common perturbation directions via SVD and Dual-Harmony Weight Orchestrator (D-HARMO) to balance intra-domain coherence with inter-domain divergence for dynamic weighting. The method yields significant transferability improvements across CIFAR-10/100 and ImageNet, outperforming state-of-the-art ensemble approaches and boosting attack success against diverse black-box models. By reframing ensemble attacks through domain generalization, HEAT offers a principled, scalable path to more transferable adversarial perturbations with practical implications for security research and defense planning.

Abstract

The development of model ensemble attacks has significantly improved the transferability of adversarial examples, but this progress also poses severe threats to the security of deep neural networks. Existing methods, however, face two critical challenges: insufficient capture of shared gradient directions across models and a lack of adaptive weight allocation mechanisms. To address these issues, we propose a novel method Harmonized Ensemble for Adversarial Transferability (HEAT), which introduces domain generalization into adversarial example generation for the first time. HEAT consists of two key modules: Consensus Gradient Direction Synthesizer, which uses Singular Value Decomposition to synthesize shared gradient directions; and Dual-Harmony Weight Orchestrator which dynamically balances intra-domain coherence, stabilizing gradients within individual models, and inter-domain diversity, enhancing transferability across models. Experimental results demonstrate that HEAT significantly outperforms existing methods across various datasets and settings, offering a new perspective and direction for adversarial attack research.

Harmonizing Intra-coherence and Inter-divergence in Ensemble Attacks for Adversarial Transferability

TL;DR

HEAT addresses the limited capture of shared gradient directions and the lack of adaptive weighting in ensemble adversarial attacks by injecting domain generalization into the attack design. It introduces Consensus Gradient Direction Synthesizer (C-GRADS) to extract common perturbation directions via SVD and Dual-Harmony Weight Orchestrator (D-HARMO) to balance intra-domain coherence with inter-domain divergence for dynamic weighting. The method yields significant transferability improvements across CIFAR-10/100 and ImageNet, outperforming state-of-the-art ensemble approaches and boosting attack success against diverse black-box models. By reframing ensemble attacks through domain generalization, HEAT offers a principled, scalable path to more transferable adversarial perturbations with practical implications for security research and defense planning.

Abstract

The development of model ensemble attacks has significantly improved the transferability of adversarial examples, but this progress also poses severe threats to the security of deep neural networks. Existing methods, however, face two critical challenges: insufficient capture of shared gradient directions across models and a lack of adaptive weight allocation mechanisms. To address these issues, we propose a novel method Harmonized Ensemble for Adversarial Transferability (HEAT), which introduces domain generalization into adversarial example generation for the first time. HEAT consists of two key modules: Consensus Gradient Direction Synthesizer, which uses Singular Value Decomposition to synthesize shared gradient directions; and Dual-Harmony Weight Orchestrator which dynamically balances intra-domain coherence, stabilizing gradients within individual models, and inter-domain diversity, enhancing transferability across models. Experimental results demonstrate that HEAT significantly outperforms existing methods across various datasets and settings, offering a new perspective and direction for adversarial attack research.
Paper Structure (13 sections, 24 equations, 3 figures, 4 tables, 1 algorithm)

This paper contains 13 sections, 24 equations, 3 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introdution
  2. Related work
  3. Methodology
  4. General Overview
  5. Harmonized Ensemble for Adversarial Transferability
  6. Consensus Gradient Direction Synthesizer
  7. Dual-Harmony Weight Orchestrator
  8. Experiments
  9. Experimental Setting
  10. Main Result
  11. Ablation Studies
  12. Further Analysis
  13. Conclusion

Figures (3)

  • Figure 1: (a) Ens averages gradients from multiple surrogate models to determine the attack direction, but its transferability is limited due to the absence of dynamic weight adjustment. (b) HEAT synthesizes a common gradient direction and dynamically adjusts the weights of gradients from different surrogate models, significantly improving the attack transferability.
  • Figure 2: An overview of HEAT: Components (b) and (c) collectively form the D-HARMO module. In (a), we use SVD to synthesize shared gradient directions; (b) captures synergistic contributions to ensure consistency in gradient directions across models; and (c) dynamically adjusts weights based on loss and alignment contributions among the models. The refined gradients generated by these components are integrated to produce highly effective adversarial images.
  • Figure 3: Visualization of adversarial attacks (a) and their impact on attribution maps for Resnet-18 (b) and WideResnet-50 (c) models using Grad-CAM.