3DMambaComplete: Exploring Structured State Space Model for Point Cloud Completion

TL;DR

3DMambaComplete introduces a 3D Structured State Space Model–based framework for point cloud completion that generates Hyperpoints, disperses them via learned offsets, and deforms a 2D grid into a detailed 3D reconstruction. Leveraging a Mamba Encoder and cross-attention, along with a HyperPoint Spread and a deformable 3D pipeline, the method attains linear complexity while capturing both local and global context. Extensive experiments on PCN, KITTI, ShapeNet55/34/Unseen21 show consistent improvements over state-of-the-art methods in CD and F-score, including real-world scans. The work highlights the effectiveness of Selective SSMs for robust 3D shape completion and detailed reconstructions in challenging, highly incomplete scenarios.

Abstract

Point cloud completion aims to generate a complete and high-fidelity point cloud from an initially incomplete and low-quality input. A prevalent strategy involves leveraging Transformer-based models to encode global features and facilitate the reconstruction process. However, the adoption of pooling operations to obtain global feature representations often results in the loss of local details within the point cloud. Moreover, the attention mechanism inherent in Transformers introduces additional computational complexity, rendering it challenging to handle long sequences effectively. To address these issues, we propose 3DMambaComplete, a point cloud completion network built on the novel Mamba framework. It comprises three modules: HyperPoint Generation encodes point cloud features using Mamba's selection mechanism and predicts a set of Hyperpoints. A specific offset is estimated, and the down-sampled points become HyperPoints. The HyperPoint Spread module disperses these HyperPoints across different spatial locations to avoid concentration. Finally, a deformation method transforms the 2D mesh representation of HyperPoints into a fine-grained 3D structure for point cloud reconstruction. Extensive experiments conducted on various established benchmarks demonstrate that 3DMambaComplete surpasses state-of-the-art point cloud completion methods, as confirmed by qualitative and quantitative analyses.

Figures (8)

• Figure 1: An overview of 3DMambaComplete. 3DMambaComplete mainly consists of three modules: (1) Given an incomplete point cloud $\mathbf{\mathit{P}}$, mamba encoder exploits coordinates $\mathcal{P}_p$ and features $\mathcal{F}_p$ of the down-sampled points to generate hyperpoints $\mathcal{P}_s$ and obtain the corresponding feature $\mathcal{F}_s$. (2) HyperPoint Spread Module. This module leverages enhanced features of $\mathcal{P}_s$ and $\mathcal{F}_s$ to broadcast hyperpoint. (3) Point Deformation. In this module, the downsampled points and hyperpoints are converted into a new set of hyperpoints using the mamba decoder. Then, a deformation module is responsible for converting the newly created hyperpoints from a 2D grid to a 3D structure.
• Figure 2: Details of HyperPoint Generation.
• Figure 3: Detailed structure for HyperPoint Spread Module.
• Figure 4: The pipeline of the Point Deformation.
• Figure 5: Visualization comparison of point cloud completion on PCN yuan2018pcn dataset.