MRG: A Multi-Robot Manufacturing Digital Scene Generation Method Using Multi-Instance Point Cloud Registration

TL;DR

This paper tackles the reliability gap between physical manufacturing environments and their digital twins by introducing MRG, a Multi-Robot Manufacturing Digital Scene Generation method that performs multi-instance point cloud registration tailored to industrial robots. The approach integrates an Instance-Focused Transformer for robust coarse-to-fine correspondence extraction, an Instance Hypothesis Generation module to recover target-instance geometry, and an Instance Filtering and Optimization stage to produce final $N_c$ rigid transformations via weighted SVD. Experimental results on Welding-Station and Scan2CAD datasets show substantial improvements over state-of-the-art multi-instance registration methods, with notable gains in MR and MP (e.g., up to ~24% improvements on Welding-Station and ~37% on Scan2CAD) and competitive runtimes, demonstrating enhanced accuracy and practicality for industrial digital scene generation. By enabling more faithful digital simulations, MRG advances the deployment of digital twins in manufacturing, potentially reducing mismatch-induced costs and accelerating optimization of real-world production lines.

Abstract

A high-fidelity digital simulation environment is crucial for accurately replicating physical operational processes. However, inconsistencies between simulation and physical environments result in low confidence in simulation outcomes, limiting their effectiveness in guiding real-world production. Unlike the traditional step-by-step point cloud "segmentation-registration" generation method, this paper introduces, for the first time, a novel Multi-Robot Manufacturing Digital Scene Generation (MRG) method that leverages multi-instance point cloud registration, specifically within manufacturing scenes. Tailored to the characteristics of industrial robots and manufacturing settings, an instance-focused transformer module is developed to delineate instance boundaries and capture correlations between local regions. Additionally, a hypothesis generation module is proposed to extract target instances while preserving key features. Finally, an efficient screening and optimization algorithm is designed to refine the final registration results. Experimental evaluations on the Scan2CAD and Welding-Station datasets demonstrate that: (1) the proposed method outperforms existing multi-instance point cloud registration techniques; (2) compared to state-of-the-art methods, the Scan2CAD dataset achieves improvements in MR and MP by 12.15% and 17.79%, respectively; and (3) on the Welding-Station dataset, MR and MP are enhanced by 16.95% and 24.15%, respectively. This work marks the first application of multi-instance point cloud registration in manufacturing scenes, significantly advancing the precision and reliability of digital simulation environments for industrial applications.

Figures (30)

• Figure 1: Comparison of different methods for generating digital welding station. The target instance point clouds in the station are shown in blue, the source point cloud after the transformation estimate is shown in yellow, point clouds similar to the target instances are shown in purple, and outlier points are shown in gray. The green bounding boxes represent the actual poses of the instances in the target point clouds, and the red bounding boxes represent the predicted poses. Instances surrounded by both red and green bounding boxes indicate successful detection, while those surrounded only by green bounding boxes indicate missed detection.
• Figure 2: The pipeline of the proposed MRG for multi-instance point cloud registration. It takes putative correspondences and the original point cloud as input, and outputs $N_{c}$ rigid transformations.
• Figure 3: Schematic characterization of different robot instances.
• Figure 4: Structure of the Instance-Focused module.
• Figure 5: Structure of the Neighbor mask module. The left is the Neighbor mask module, and the right is the standard self-attention module.