epsilon-Mesh Attack: A Surface-based Adversarial Point Cloud Attack for Facial Expression Recognition

TL;DR

An adversarial attack called $\epsilon$-Mesh Attack, which operates on point cloud data via limiting perturbations to be on the mesh surface, which has tighter perturbation bounds compared to $L_2$ and $L_\infty$ norm bounded attacks that operate on unit-ball.

Abstract

Point clouds and meshes are widely used 3D data structures for many computer vision applications. While the meshes represent the surfaces of an object, point cloud represents sampled points from the surface which is also the output of modern sensors such as LiDAR and RGB-D cameras. Due to the wide application area of point clouds and the recent advancements in deep neural networks, studies focusing on robust classification of the 3D point cloud data emerged. To evaluate the robustness of deep classifier networks, a common method is to use adversarial attacks where the gradient direction is followed to change the input slightly. The previous studies on adversarial attacks are generally evaluated on point clouds of daily objects. However, considering 3D faces, these adversarial attacks tend to affect the person's facial structure more than the desired amount and cause malformation. Specifically for facial expressions, even a small adversarial attack can have a significant effect on the face structure. In this paper, we suggest an adversarial attack called $ε$-Mesh Attack, which operates on point cloud data via limiting perturbations to be on the mesh surface. We also parameterize our attack by $ε$ to scale the perturbation mesh. Our surface-based attack has tighter perturbation bounds compared to $L_2$ and $L_\infty$ norm bounded attacks that operate on unit-ball. Even though our method has additional constraints, our experiments on CoMA, Bosphorus and FaceWarehouse datasets show that $ε$-Mesh Attack (Perpendicular) successfully confuses trained DGCNN and PointNet models $99.72\%$ and $97.06\%$ of the time, with indistinguishable facial deformations. The code is available at https://github.com/batuceng/e-mesh-attack.

Figures (8)

• Figure 1: An example Face mesh from CoMACOMA:ECCV18 dataset and the suggested triangular bounds for the surface preserving white box attack scaled by parameter $\epsilon$.
• Figure 2: Division of the area near a selected triangle to calculate perpendicular projection.
• Figure 3: Projection example for adversarial perturbation $\nabla$ (left). Different projection methods in right: PGDsun2021adversarially, Central and Perpendicular from top to bottom.
• Figure 4: Example front-viewed point cloud images from different datasets are given at each row. Predictions obtained from model denoted at left are written at the top of each image with green and red colors for correct and incorrect predictions respectively. The columns represent a clean face, and its attacked versions by PGD, PGD-L2, $\epsilon$-mesh central projection, and $\epsilon$-mesh perpendicular projection from left to right. Also, distances between the attacked and clean point clouds are denoted below with each sample for $L_2$ and Chamfer distances.
• Figure 5: Adversarial attack results for varying number of steps (left column) and epsilon values (right column). The top and bottom rows show results on PointNet and DGCNN respectively.