PointMT: Efficient Point Cloud Analysis with Hybrid MLP-Transformer Architecture

TL;DR

This study tackles the quadratic complexity of the self-attention mechanism by introducing a linear complexity local attention mechanism for effective feature aggregation, and proposes an efficient point cloud analysis architecture, Point MLP-Transformer (PointMT).

Abstract

In recent years, point cloud analysis methods based on the Transformer architecture have made significant progress, particularly in the context of multimedia applications such as 3D modeling, virtual reality, and autonomous systems. However, the high computational resource demands of the Transformer architecture hinder its scalability, real-time processing capabilities, and deployment on mobile devices and other platforms with limited computational resources. This limitation remains a significant obstacle to its practical application in scenarios requiring on-device intelligence and multimedia processing. To address this challenge, we propose an efficient point cloud analysis architecture, \textbf{Point} \textbf{M}LP-\textbf{T}ransformer (PointMT). This study tackles the quadratic complexity of the self-attention mechanism by introducing a linear complexity local attention mechanism for effective feature aggregation. Additionally, to counter the Transformer's focus on token differences while neglecting channel differences, we introduce a parameter-free channel temperature adaptation mechanism that adaptively adjusts the attention weight distribution in each channel, enhancing the precision of feature aggregation. To improve the Transformer's slow convergence speed due to the limited scale of point cloud datasets, we propose an MLP-Transformer hybrid module, which significantly enhances the model's convergence speed. Furthermore, to boost the feature representation capability of point tokens, we refine the classification head, enabling point tokens to directly participate in prediction. Experimental results on multiple evaluation benchmarks demonstrate that PointMT achieves performance comparable to state-of-the-art methods while maintaining an optimal balance between performance and accuracy.

Figures (9)

• Figure 1: An overview of the PointMT network architecture and its key components. The diagram illustrates the TA-Attention module, which integrates a linear local attention mechanism (see Sec. \ref{['sec-linear-attn']}) and a channel temperatureadaptation strategy (see Sec. \ref{['sec-temp-adapt']}) to enhance feature representation. Additionally, it features the MT-Block, an innovative hybrid unit that combines MLP and Transformer architectures (see Sec. \ref{['sec-MT-hybrid']}), as well as the SPF Cls. Head (see Sec. \ref{['sec-SPF-Head']}), a novel classification head designed based on the shape-point-fusion mechanism to improve classification accuracy.
• Figure 2: Local attention with linear complexity proportional to the number of points $N$ and the size of local neighborhoods $k$.
• Figure 3: Comparison of convergence rates among different model architectures (M: MLP, A: Attention, H: MLP-Transformer hybrid). The hybrid architecture achieves notable accuracies of $93.4\%$ and $94.2\%$ after 30 and 60 epochs, respectively, indicating rapid convergence.
• Figure 4: t-SNE visualization of encoder features using (a) the conventional classification head and (b) the SPF classification head.
• Figure 5: t-SNE visualization of logit outputs using (a) the conventional classification head and (b) the SPF classification head.