Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Gradient Structure Estimation under Label-Only Oracles via Spectral Sensitivity

Jun Liu, Leo Yu Zhang, Fengpeng Li, Isao Echizen, Jiantao Zhou

TL;DR

The paper tackles the challenge of gradient estimation in hard-label black-box attacks by uncovering a theoretical bridge between discrete sign-search methods and gradient sign recovery. It introduces DPAttack, a two-stage framework that (i) uses a frequency-domain prior via Block-DCT sensitivity to obtain a zero-query, structured initialization and (ii) applies Pattern-Driven Optimization to preserve spatial coherence and accelerate convergence. The authors provide theoretical guarantees showing that RayS-style discrete searches can approximate FGSM under certain regimes, and they prove per-query sign-alignment advantages for the pattern-driven tree over naive dyadic searches. Empirically, DPAttack achieves superior attack success rates and markedly lower query costs across CIFAR-10, ImageNet, CLIP, and commercial APIs, while also bypassing defenses like Blacklight and generalizing to dense prediction tasks. The work offers significant implications for evaluating and improving robustness of real-world systems against efficient hard-label adversaries.

Abstract

Hard-label black-box settings, where only top-1 predicted labels are observable, pose a fundamentally constrained yet practically important feedback model for understanding model behavior. A central challenge in this regime is whether meaningful gradient information can be recovered from such discrete responses. In this work, we develop a unified theoretical perspective showing that a wide range of existing sign-flipping hard-label attacks can be interpreted as implicitly approximating the sign of the true loss gradient. This observation reframes hard-label attacks from heuristic search procedures into instances of gradient sign recovery under extremely limited feedback. Motivated by this first-principles understanding, we propose a new attack framework that combines a zero-query frequency-domain initialization with a Pattern-Driven Optimization (PDO) strategy. We establish theoretical guarantees demonstrating that, under mild assumptions, our initialization achieves higher expected cosine similarity to the true gradient sign compared to random baselines, while the proposed PDO procedure attains substantially lower query complexity than existing structured search approaches. We empirically validate our framework through extensive experiments on CIFAR-10, ImageNet, and ObjectNet, covering standard and adversarially trained models, commercial APIs, and CLIP-based models. The results show that our method consistently surpasses SOTA hard-label attacks in both attack success rate and query efficiency, particularly in low-query regimes. Beyond image classification, our approach generalizes effectively to corrupted data, biomedical datasets, and dense prediction tasks. Notably, it also successfully circumvents Blacklight, a SOTA stateful defense, resulting in a $0\%$ detection rate. Our code will be released publicly soon at https://github.com/csjunjun/DPAttack.git.

Gradient Structure Estimation under Label-Only Oracles via Spectral Sensitivity

TL;DR

The paper tackles the challenge of gradient estimation in hard-label black-box attacks by uncovering a theoretical bridge between discrete sign-search methods and gradient sign recovery. It introduces DPAttack, a two-stage framework that (i) uses a frequency-domain prior via Block-DCT sensitivity to obtain a zero-query, structured initialization and (ii) applies Pattern-Driven Optimization to preserve spatial coherence and accelerate convergence. The authors provide theoretical guarantees showing that RayS-style discrete searches can approximate FGSM under certain regimes, and they prove per-query sign-alignment advantages for the pattern-driven tree over naive dyadic searches. Empirically, DPAttack achieves superior attack success rates and markedly lower query costs across CIFAR-10, ImageNet, CLIP, and commercial APIs, while also bypassing defenses like Blacklight and generalizing to dense prediction tasks. The work offers significant implications for evaluating and improving robustness of real-world systems against efficient hard-label adversaries.

Abstract

Hard-label black-box settings, where only top-1 predicted labels are observable, pose a fundamentally constrained yet practically important feedback model for understanding model behavior. A central challenge in this regime is whether meaningful gradient information can be recovered from such discrete responses. In this work, we develop a unified theoretical perspective showing that a wide range of existing sign-flipping hard-label attacks can be interpreted as implicitly approximating the sign of the true loss gradient. This observation reframes hard-label attacks from heuristic search procedures into instances of gradient sign recovery under extremely limited feedback. Motivated by this first-principles understanding, we propose a new attack framework that combines a zero-query frequency-domain initialization with a Pattern-Driven Optimization (PDO) strategy. We establish theoretical guarantees demonstrating that, under mild assumptions, our initialization achieves higher expected cosine similarity to the true gradient sign compared to random baselines, while the proposed PDO procedure attains substantially lower query complexity than existing structured search approaches. We empirically validate our framework through extensive experiments on CIFAR-10, ImageNet, and ObjectNet, covering standard and adversarially trained models, commercial APIs, and CLIP-based models. The results show that our method consistently surpasses SOTA hard-label attacks in both attack success rate and query efficiency, particularly in low-query regimes. Beyond image classification, our approach generalizes effectively to corrupted data, biomedical datasets, and dense prediction tasks. Notably, it also successfully circumvents Blacklight, a SOTA stateful defense, resulting in a detection rate. Our code will be released publicly soon at https://github.com/csjunjun/DPAttack.git.
Paper Structure (54 sections, 18 theorems, 60 equations, 15 figures, 19 tables, 3 algorithms)

This paper contains 54 sections, 18 theorems, 60 equations, 15 figures, 19 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Formulation & Background
  4. Hard-Label Black-Box Attacks
  5. Discrete Hard-Label Attacks
  6. Theoretical Foundations: From Discrete Search to Gradient Sign Estimation
  7. Our Proposed Attack
  8. Threat Model
  9. The Overall Attack Pipeline of DPAttack
  10. Better Initialization via Dynamic Decision-Making (DDM)
  11. Analysis: Block-DCT wallace1991jpeg Frequency Sensitivity (BFS)
  12. Methodology: Constructing Initial Directions
  13. Validation and Dynamic Selection
  14. Pattern-Driven Optimization (PDO)
  15. Algorithm Implementation
  16. ...and 39 more sections

Key Result

Theorem 1

Let $\mathbf{u}\in\mathbb{S}^{d-1}$ be an arbitrary unit vector. Let $\{\mathbf{d}_j\}_{j=1}^m$ be $m$ vectors sampled independently and uniformly from the binary hypercube $\mathcal{H}\equiv\{-1,+1\}^d$, and let $\hat{\mathbf{d}}_j=\mathbf{d}_j/{\sqrt{d}}$ be their normalized counterparts. For any

Figures (15)

  • Figure 1: Comparison of our approach with traditional methods. Our method achieves a better initialization $\mathbf{d}_0$ with smaller angular deviation $\theta_0$ and boundary distance $g(\mathbf{d_0})$ compared to the traditional initialization $\bar{\mathbf{d}}_0$. Furthermore, it employs a more query-efficient strategy to approximate the ideal descent direction $\mathbf{d}^*$.
  • Figure 2: Framework of the proposed hard-label attack method DPAttack. (a) Stage 1: The Dynamic Decision-Making (DDM) module generates a superior initial perturbation direction $\mathbf{d}_0$. (b) Stage 2: The Pattern-Driven Optimization (PDO) module iteratively refines the direction and the perturbation magnitude $r$, yielding the final adversarial example $\mathbf{x}^*=\operatorname{clip}(\mathbf{x}+r\hat{\mathbf{d}}^*,[0,1])$.
  • Figure 3: Classification sensitivity analysis of frequency bands via the proposed BFS. Higher CE loss indicates greater sensitivity of the corresponding BDCT frequency band. The BDCT block size is set to $w=8$.
  • Figure 4: Clean image frequency statistics. The solid line and the shaded region represent the mean and standard deviation of the log-variance $\log(\sigma_{c,i,j}^2+1)$ (defined in Eq. (\ref{['eq:d1_1']})).
  • Figure 5: (a) Cosine similarity between the true gradient sign $\text{sgn}\nabla\mathcal{L}(\mathbf{x},y)$ and its block-wise averaged approximations. (b) Cosine similarity between the true gradient sign and directions from various initialization methods.
  • ...and 10 more figures

Theorems & Definitions (20)

  • Theorem 1: Approximation Lower Bound for NRayS
  • Definition 1: Subset-wise Directional Derivative Influence
  • Lemma 1
  • Theorem 2: Gradient Approximation Guarantee for HRayS
  • Theorem 3: Positive Expected Alignment under Frequency-Variance Prior
  • Theorem 4: Per-query Sign Alignment Dominance
  • Theorem 5: Query Complexity under Block Sign-Coherence
  • Theorem 1: Approximation Lower Bound for NRayS
  • Definition 1: Subset-wise Directional Derivative Influence
  • Lemma 1
  • ...and 10 more