Hard-Label Black-Box Attacks on 3D Point Clouds

TL;DR

This work introduces a novel 3D attack method based on a new spectrum-aware decision boundary algorithm to generate high-quality adversarial samples, which competitively outperforms existing white/black-box attackers in terms of attack performance and adversary quality.

Abstract

With the maturity of depth sensors in various 3D safety-critical applications, 3D point cloud models have been shown to be vulnerable to adversarial attacks. Almost all existing 3D attackers simply follow the white-box or black-box setting to iteratively update coordinate perturbations based on back-propagated or estimated gradients. However, these methods are hard to deploy in real-world scenarios (no model details are provided) as they severely rely on parameters or output logits of victim models. To this end, we propose point cloud attacks from a more practical setting, i.e., hard-label black-box attack, in which attackers can only access the prediction label of 3D input. We introduce a novel 3D attack method based on a new spectrum-aware decision boundary algorithm to generate high-quality adversarial samples. In particular, we first construct a class-aware model decision boundary, by developing a learnable spectrum-fusion strategy to adaptively fuse point clouds of different classes in the spectral domain, aiming to craft their intermediate samples without distorting the original geometry. Then, we devise an iterative coordinate-spectrum optimization method with curvature-aware boundary search to move the intermediate sample along the decision boundary for generating adversarial point clouds with trivial perturbations. Experiments demonstrate that our attack competitively outperforms existing white/black-box attackers in terms of attack performance and adversary quality.

Figures (11)

• Figure 1: Illustration of our motivation. Our 3D hard-label black-box setting cannot access any model details of the hidden layer or the logits of the output layer. To tackle this setting, we develop a spectrum-aware decision boundary algorithm to first fuse point clouds in the spectral domain for generating the class-aware decision boundary, and then iteratively optimize the best adversarial sample along the decision boundary based on its geometric curvature information and distances.
• Figure 2: Overall pipeline of our proposed Hard-Label Black-Box Attack. Specifically, we first design a learnable spectrum-fusion boundary-cloud generation module to fuse the source cloud with a set of target clouds in the spectral domain to construct candidate clouds with high quality. Then, we utilize the obtained candidate clouds to construct the decision boundary and select the best one to project onto the decision boundary for obtaining the initial boundary cloud. After that, we introduce a geometry-aware boundary-cloud optimization module to iteratively move this boundary cloud along the decision boundary via a joint coordinate-spectrum iterative walking strategy to achieve the best place with smallest distortion. Here, we consider the curvature information of the decision boundary to help update the boundary cloud. The well-optimized boundary cloud is our final generated adversarial point cloud.
• Figure 3: Illustration of how we generate the learnable spectrum-fusion rates. We first design a discriminator to distinguish the benign and randomly fused point clouds. Then, we initial a learnable rate $\mathbf{\alpha}$ and utilize the gradients of discriminator to backpropagate and update it.
• Figure 4: Illustration of why we propose to optimize the boundary cloud in both coordinate and spectrum domains. The boundary cloud may have different coordinate and spectrum distances from the original cloud. Therefore, the point cloud in the convex position of the coordinate domain may no longer be a convex position in the spectrum domain and can be well-optimized. In this manner, we can avoid the local optimal problem in the coordinate domain by additionally optimizing the cloud in the spectrum domain.
• Figure 5: Illustration on our geometry-aware optimization strategy. Specifically, given the cloud $\mathbf{P}_b^{(t)}$, we iteratively update and adjust the direction $\mathbf{\xi}_t$ to search the cloud $\mathbf{P}_c$ along the semicircular path until reaching the best adversarial position $\mathbf{P}_b^{(t+1)}$.