Quantifying the Risk of Transferred Black Box Attacks

TL;DR

The paper tackles the problem of quantifying risk from transferred black-box adversarial attacks in security-sensitive AI, where full adversarial risk mapping is impractical due to the large input space and transferability. It proposes a targeted resilience-testing framework that selects surrogate models based on Centered Kernel Alignment (CKA) similarity to the target model, combining high- and low-similarity surrogates to cover diverse adversarial subspaces. Risk estimation is performed with regression-based estimators to provide realistic, action-oriented risk metrics suitable for regulatory compliance. The work also formalizes surrogate selection with thresholds $r_{1}$ and $r_{2}$ and discusses DBS-based refinements, outlining practical steps and future directions toward architecture-derived similarity measures.

Abstract

Neural networks have become pervasive across various applications, including security-related products. However, their widespread adoption has heightened concerns regarding vulnerability to adversarial attacks. With emerging regulations and standards emphasizing security, organizations must reliably quantify risks associated with these attacks, particularly regarding transferred adversarial attacks, which remain challenging to evaluate accurately. This paper investigates the complexities involved in resilience testing against transferred adversarial attacks. Our analysis specifically addresses black-box evasion attacks, highlighting transfer-based attacks due to their practical significance and typically high transferability between neural network models. We underline the computational infeasibility of exhaustively exploring high-dimensional input spaces to achieve complete test coverage. As a result, comprehensive adversarial risk mapping is deemed impractical. To mitigate this limitation, we propose a targeted resilience testing framework that employs surrogate models strategically selected based on Centered Kernel Alignment (CKA) similarity. By leveraging surrogate models exhibiting both high and low CKA similarities relative to the target model, the proposed approach seeks to optimize coverage of adversarial subspaces. Risk estimation is conducted using regression-based estimators, providing organizations with realistic and actionable risk quantification.

Figures (1)

• Figure 1: Conceptual illustration of adversarial subspace overlap across models with varying similarity. Target model (yellow) and potential surrogate models (a–e), each with its own adversarial subspace.