Robust 3D Point Clouds Classification based on Declarative Defenders

TL;DR

This paper explores three distinct algorithms for mapping 3D point clouds into 2D images and scrutinizes the feature disparities between regular 2D images and projected 2D images to demonstrate superior accuracy and robustness against adversarial attacks.

Abstract

3D point cloud classification requires distinct models from 2D image classification due to the divergent characteristics of the respective input data. While 3D point clouds are unstructured and sparse, 2D images are structured and dense. Bridging the domain gap between these two data types is a non-trivial challenge to enable model interchangeability. Recent research using Lattice Point Classifier (LPC) highlights the feasibility of cross-domain applicability. However, the lattice projection operation in LPC generates 2D images with disconnected projected pixels. In this paper, we explore three distinct algorithms for mapping 3D point clouds into 2D images. Through extensive experiments, we thoroughly examine and analyze their performance and defense mechanisms. Leveraging current large foundation models, we scrutinize the feature disparities between regular 2D images and projected 2D images. The proposed approaches demonstrate superior accuracy and robustness against adversarial attacks. The generative model-based mapping algorithms yield regular 2D images, further minimizing the domain gap from regular 2D classification tasks. The source code is available at https://github.com/KaidongLi/pytorch-LatticePointClassifier.git.

Figures (7)

• Figure 1: Illustration of mapping algorithms to transform 3D point clouds to 2D images.
• Figure 2: Projection Classifier. Input point clouds are projected onto the $x-y$ axis plane by removing $z$-axis information. The coordinates of projected points are rounded using the floor function.
• Figure 3: Graph Drawing Classifier. Input point clouds are divided into 32 clusters using balanced KMeans clustering lyu2020learning. Delaunay triangulation delaunay1934sphere is applied on the 32 cluster centers and within each individual cluster, generating 2-level graphs. Then the top-level graph is mapped to a $16 \times 16$ grid, where each cluster center occupies a grid cell. Then each grid cell is filled with a lower-level $16 \times 16$ grid, obtained from within-cluster graphs, yielding the final $256 \times 256$ image.
• Figure 4: Rendering Classifier. Input point clouds are converted to a 2D depth map using z-buffer projection. Positional embeddings are appended to the depth map to boost global context metzer2022z2p. Then the generated features are fed to a modified U-Net ronneberger2015u to produce final rendered images. In the modified U-Net, the normalization layer is replaced with a style-based adaptive instance normalization (AdaIN) karras2019style, which also takes scene control features to influence output image style.
• Figure 5: Visualization of 2,468 ModelNet40 wu20153d test point clouds using t-SNE van2008visualizing. Each plot is based on the DNN features generated by corresponding mapping algorithms before linear layers.