Fourier Decomposition for Explicit Representation of 3D Point Cloud Attributes

TL;DR

This paper tackles the challenge of effectively modeling colored 3D point clouds by introducing a Fourier-based encoding that disentangles color and geometry via amplitude $\mathcal{A}$ and phase $\mathcal{P}$ obtained from a 3D DFT on a voxelized grid with occupancy $\pi$. By leveraging spectral-domain operations, the method achieves a large receptive field and enables independent learning of color and geometry, leading to strong performance in classification and high-quality style transfer, as well as a simple yet effective data augmentation strategy based on amplitude swapping. The authors validate their approach on the DensePoint dataset, demonstrating state-of-the-art classification accuracy ($OA$ up to $98.43\%$) and compelling qualitative style-transfer results, supported by ablations and qualitative analyses. The work contributes a principled, geometry-color disentanglement framework for colored point clouds and showcases its practical impact across multiple 3D vision tasks, with potential for broader applications beyond recognition.

Abstract

While 3D point clouds are widely utilized across various vision applications, their irregular and sparse nature make them challenging to handle. In response, numerous encoding approaches have been proposed to capture the rich semantic information of point clouds. Yet, a critical limitation persists: a lack of consideration for colored point clouds which are more capable 3D representations as they contain diverse attributes: color and geometry. While existing methods handle these attributes separately on a per-point basis, this leads to a limited receptive field and restricted ability to capture relationships across multiple points. To address this, we pioneer a point cloud encoding methodology that leverages 3D Fourier decomposition to disentangle color and geometric features while extending the receptive field through spectral-domain operations. Our analysis confirms that this encoding approach effectively separates feature components, where the amplitude uniquely captures color attributes and the phase encodes geometric structure, thereby enabling independent learning and utilization of both attributes. Furthermore, the spectral-domain properties of these components naturally aggregate local features while considering multiple points' information. We validate our point cloud encoding approach on point cloud classification and style transfer tasks, achieving state-of-the-art results on the DensePoint dataset with improvements via a proposed amplitude-based data augmentation strategy.

Figures (14)

• Figure 1: We present a Fourier-based point cloud encoding method that explicitly leverages amplitude and phase to represent the color and geometric attributes of the point cloud. This encoding enables effective processing across various point cloud tasks, including classification, style transfer, and data augmentation.
• Figure 2: Detailed process of 3D Fourier decomposition and reconstruction. (a) The point cloud is voxelized and decomposed via Fourier Transform into amplitude and phase. An additional channel $\pi$ represents the probability of a point existing in each voxel. (b) The inverse Fourier Transform reconstructs voxel data, removing low-$\pi$ voxels to reduce amplitude-phase misalignment noise.
• Figure 3: An experimental investigation into Fourier decomposition in point clouds reveals the distinct roles of amplitude and phase. Performing Fourier decomposition separately on two point clouds and exchanging their amplitude components before reconstruction results in a transfer of color attributes. In contrast, exchanging phase components alters the geometric structure, indicating that amplitude primarily encodes color information, whereas phase captures the underlying spatial arrangement.
• Figure 4: An overview of our classification model using Fourier-based point encoding. Input points are decomposed into amplitude and phase through Fourier decomposition, then separately processed by a color encoder $\mathcal{E}_\text{col}$ capturing color attributes and a geometry encoder $\mathcal{E}_\text{geo}$ capturing geometric attributes. The resulting feature vectors are then concatenated and computed through a fusion module $\mathcal{M}_\text{fus}$ to generate the final feature representation.
• Figure 5: Overall pipeline of our point cloud encoding method applied to style transfer. The stylized point cloud's phase is initialized to be identical to that of the content point cloud $\mathbf{P}_\text{content}$, while its amplitude is initialized through a linear interpolation between the amplitudes of the content and style point clouds. Subsequently, features are extracted from the amplitudes of the style, content, and stylized point clouds using a color encoder $\mathcal{E}_\text{col}$. The loss is then computed, and the amplitude of the stylized point cloud is iteratively updated. Once the update process is complete, the final stylized point cloud is generated by performing Fourier reconstruction using the refined amplitude and phase.