Efficient Point Transformer with Dynamic Token Aggregating for LiDAR Point Cloud Processing

TL;DR

LiDAR point cloud segmentation with 3D Transformers is challenged by heavy computation from dense attention and slow sampling. The authors propose DTA-Former, a dynamic token framework with Learnable Token Sparsification (LTS), Dynamic Token Aggregating (DTA), Global Feature Enhancement (GFE), and Iterative Token Reconstruction (ITR), organized in a two-stage W-net. LTS adaptively selects key tokens using local-global context; DTA aggregates features across the full token set via Weighted Cross-Attention; GFE enhances long-range features with dual attention; ITR reconstructs dense predictions by propagating semantic relations back to the original tokens. Experiments on MS-LiDAR, DALES, and ShapeNet show state-of-the-art accuracy and efficiency, highlighting the method's scalability for large-scale LiDAR datasets.

Abstract

Recently, LiDAR point cloud processing and analysis have made great progress due to the development of 3D Transformers. However, existing 3D Transformer methods usually are computationally expensive and inefficient due to their huge and redundant attention maps. They also tend to be slow due to requiring time-consuming point cloud sampling and grouping processes. To address these issues, we propose an efficient point TransFormer with Dynamic Token Aggregating (DTA-Former) for point cloud representation and processing. Firstly, we propose an efficient Learnable Token Sparsification (LTS) block, which considers both local and global semantic information for the adaptive selection of key tokens. Secondly, to achieve the feature aggregation for sparsified tokens, we present the first Dynamic Token Aggregating (DTA) block in the 3D Transformer paradigm, providing our model with strong aggregated features while preventing information loss. After that, a dual-attention Transformer-based Global Feature Enhancement (GFE) block is used to improve the representation capability of the model. Equipped with LTS, DTA, and GFE blocks, DTA-Former achieves excellent classification results via hierarchical feature learning. Lastly, a novel Iterative Token Reconstruction (ITR) block is introduced for dense prediction whereby the semantic features of tokens and their semantic relationships are gradually optimized during iterative reconstruction. Based on ITR, we propose a new W-net architecture, which is more suitable for Transformer-based feature learning than the common U-net design.

Figures (12)

• Figure 1: Architecture of DTA-Former. A novel W-net architecture is proposed, with two stages containing LTS, DTA, GFE, and ITR blocks. To clearly visualize the semantic homogeneous clustering results, the airplane model in ShapeNet yi2016scalable is taken as an example to illustrate the details of the method. The two WCA-maps show strong correlations between attention weights and point semantics, where the stronger the correlation, the redder the point. Query points are indicated by yellow stars.
• Figure 2: Illustration of LTS results on Stage-1. Utilizing both local and global semantic information of tokens, the token sparsification process is dynamically and efficiently updated as the computation proceeds.
• Figure 3: Visualization of sparsified tokens from LTS blocks for different object categories. As observed, LTS works well in keeping the structural skeleton of objects, even in scenarios with very sparse sampling tokens.
• Figure 4: Illustration of the DTA block on Stage-1. It exploits the long-range context information better for feature aggregating without information loss. It is more efficient than local grouping and pooling operations due to avoiding neighborhood construction.
• Figure 5: Brief illustration of the segmentation network.