Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Region-Transformer: Self-Attention Region Based Class-Agnostic Point Cloud Segmentation

Dipesh Gyawali, Jian Zhang, BB Karki

TL;DR

The paper tackles class-agnostic segmentation of 3D point clouds by introducing Region-Transformer, a region-growth framework augmented with self-attention. It merges a two-branch Point Transformer-based encoder-decoder with local neighborhood attention to iteratively grow regions via add/remove decisions trained with a binary cross-entropy loss, and it relies on synthetic, region-based supervision from S3DIS and ScanNet. The key contributions include the novel combination of self-attention with region-growth, a robust inference loop with curvature-based seeding and small-segment safeguards, and strong empirical performance on indoor datasets with scalable generalization to large scenes. This approach offers practical benefits for robotics, digital twinning, and autonomous navigation by enabling flexible, label-free instance segmentation with efficient inference.

Abstract

Point cloud segmentation, which helps us understand the environment of specific structures and objects, can be performed in class-specific and class-agnostic ways. We propose a novel region-based transformer model called Region-Transformer for performing class-agnostic point cloud segmentation. The model utilizes a region-growth approach and self-attention mechanism to iteratively expand or contract a region by adding or removing points. It is trained on simulated point clouds with instance labels only, avoiding semantic labels. Attention-based networks have succeeded in many previous methods of performing point cloud segmentation. However, a region-growth approach with attention-based networks has yet to be used to explore its performance gain. To our knowledge, we are the first to use a self-attention mechanism in a region-growth approach. With the introduction of self-attention to region-growth that can utilize local contextual information of neighborhood points, our experiments demonstrate that the Region-Transformer model outperforms previous class-agnostic and class-specific methods on indoor datasets regarding clustering metrics. The model generalizes well to large-scale scenes. Key advantages include capturing long-range dependencies through self-attention, avoiding the need for semantic labels during training, and applicability to a variable number of objects. The Region-Transformer model represents a promising approach for flexible point cloud segmentation with applications in robotics, digital twinning, and autonomous vehicles.

Region-Transformer: Self-Attention Region Based Class-Agnostic Point Cloud Segmentation

TL;DR

The paper tackles class-agnostic segmentation of 3D point clouds by introducing Region-Transformer, a region-growth framework augmented with self-attention. It merges a two-branch Point Transformer-based encoder-decoder with local neighborhood attention to iteratively grow regions via add/remove decisions trained with a binary cross-entropy loss, and it relies on synthetic, region-based supervision from S3DIS and ScanNet. The key contributions include the novel combination of self-attention with region-growth, a robust inference loop with curvature-based seeding and small-segment safeguards, and strong empirical performance on indoor datasets with scalable generalization to large scenes. This approach offers practical benefits for robotics, digital twinning, and autonomous navigation by enabling flexible, label-free instance segmentation with efficient inference.

Abstract

Point cloud segmentation, which helps us understand the environment of specific structures and objects, can be performed in class-specific and class-agnostic ways. We propose a novel region-based transformer model called Region-Transformer for performing class-agnostic point cloud segmentation. The model utilizes a region-growth approach and self-attention mechanism to iteratively expand or contract a region by adding or removing points. It is trained on simulated point clouds with instance labels only, avoiding semantic labels. Attention-based networks have succeeded in many previous methods of performing point cloud segmentation. However, a region-growth approach with attention-based networks has yet to be used to explore its performance gain. To our knowledge, we are the first to use a self-attention mechanism in a region-growth approach. With the introduction of self-attention to region-growth that can utilize local contextual information of neighborhood points, our experiments demonstrate that the Region-Transformer model outperforms previous class-agnostic and class-specific methods on indoor datasets regarding clustering metrics. The model generalizes well to large-scale scenes. Key advantages include capturing long-range dependencies through self-attention, avoiding the need for semantic labels during training, and applicability to a variable number of objects. The Region-Transformer model represents a promising approach for flexible point cloud segmentation with applications in robotics, digital twinning, and autonomous vehicles.
Paper Structure (10 sections, 3 equations, 5 figures, 3 tables)

This paper contains 10 sections, 3 equations, 5 figures, 3 tables.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORKS
  3. METHODOLOGY
  4. Problem Definition and Formulation
  5. Network Architecture
  6. Self-attention in Region-Growth
  7. Data Preparation and Simulation
  8. Inference
  9. EXPERIMENTS AND RESULTS
  10. CONCLUSIONS

Figures (5)

  • Figure 1: Region-Transformer network architecture for class-agnostic segmentation. Block B1 generating (128,128) features and B2 generating (128,256,512,1024) features act as encoders for inlier and neighbor sets. Block B3 generating (512,256,128) features acts as a decoder. Points from B1 and B2 are average-pooled, and inlier and neighbor sets are concatenated together to form a bottleneck. The encoded features are broadcasted into N number of points, and features from B1 output are concatenated to broadcasted features to get positional information of each point. In the last layer, the add and remove mask predictions are made.
  • Figure 2: Point Transformer Block.
  • Figure 3: Object-agnostic segmentation results across Scannet a) Raw point cloud original visualization (b) Ground truth original segmentation (c) PointNet++ segmentation (d) LRGNet segmentation (e) Region-Transformer (Our Method).
  • Figure 4: Object-agnostic segmentation results across S3DIS a)Raw point cloud original visualization (b) Ground-truth original segmentation(c) PointNet++ segmentation (d) LRGNet segmentation (e) Region-Transformer (Our Method).
  • Figure 5: Object-agnostic Segmentation results on real-world, large-scale factory data a) Raw point cloud original visualization (b) LRGNet segmentation (c) Region-Transformer (Our Method).