Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Coarse-to-Fine Detection of Multiple Seams for Robotic Welding

Pengkun Wei, Shuo Cheng, Dayou Li, Ran Song, Yipeng Zhang, Wei Zhang

TL;DR

This work tackles the challenge of efficiently detecting and localizing multiple weld seams with sub-millimeter accuracy. It introduces a coarse-to-fine framework that uses RGB images for coarse localization via FastSAM and region-growing on cropped 3D point clouds to extract seam edges and generate a unified 6-DOF welding path. The approach includes a cropping-based reduction of irrelevant data and a region-growing edge extraction module, validated on linear and curved seams in both lab and industrial-like settings, with real-world video demonstrations. The results show improved efficiency while maintaining accuracy, enabling practical multi-seam robotic welding on production lines.

Abstract

Efficiently detecting target weld seams while ensuring sub-millimeter accuracy has always been an important challenge in autonomous welding, which has significant application in industrial practice. Previous works mostly focused on recognizing and localizing welding seams one by one, leading to inferior efficiency in modeling the workpiece. This paper proposes a novel framework capable of multiple weld seams extraction using both RGB images and 3D point clouds. The RGB image is used to obtain the region of interest by approximately localizing the weld seams, and the point cloud is used to achieve the fine-edge extraction of the weld seams within the region of interest using region growth. Our method is further accelerated by using a pre-trained deep learning model to ensure both efficiency and generalization ability. The performance of the proposed method has been comprehensively tested on various workpieces featuring both linear and curved weld seams and in physical experiment systems. The results showcase considerable potential for real-world industrial applications, emphasizing the method's efficiency and effectiveness. Videos of the real-world experiments can be found at https://youtu.be/pq162HSP2D4.

Coarse-to-Fine Detection of Multiple Seams for Robotic Welding

TL;DR

This work tackles the challenge of efficiently detecting and localizing multiple weld seams with sub-millimeter accuracy. It introduces a coarse-to-fine framework that uses RGB images for coarse localization via FastSAM and region-growing on cropped 3D point clouds to extract seam edges and generate a unified 6-DOF welding path. The approach includes a cropping-based reduction of irrelevant data and a region-growing edge extraction module, validated on linear and curved seams in both lab and industrial-like settings, with real-world video demonstrations. The results show improved efficiency while maintaining accuracy, enabling practical multi-seam robotic welding on production lines.

Abstract

Efficiently detecting target weld seams while ensuring sub-millimeter accuracy has always been an important challenge in autonomous welding, which has significant application in industrial practice. Previous works mostly focused on recognizing and localizing welding seams one by one, leading to inferior efficiency in modeling the workpiece. This paper proposes a novel framework capable of multiple weld seams extraction using both RGB images and 3D point clouds. The RGB image is used to obtain the region of interest by approximately localizing the weld seams, and the point cloud is used to achieve the fine-edge extraction of the weld seams within the region of interest using region growth. Our method is further accelerated by using a pre-trained deep learning model to ensure both efficiency and generalization ability. The performance of the proposed method has been comprehensively tested on various workpieces featuring both linear and curved weld seams and in physical experiment systems. The results showcase considerable potential for real-world industrial applications, emphasizing the method's efficiency and effectiveness. Videos of the real-world experiments can be found at https://youtu.be/pq162HSP2D4.
Paper Structure (14 sections, 4 equations, 6 figures, 2 tables, 1 algorithm)

This paper contains 14 sections, 4 equations, 6 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Laser sensor-based Methods
  4. Other sensor-based Methods
  5. Method
  6. Problem Statement
  7. Workpiece Edge Model
  8. Coarse Seam Detection Using VLM
  9. Fine Edge Extraction Based on Region-Growing
  10. Experiments
  11. Effect of the Downsampling Parameter
  12. Quantitative Comparison
  13. Application in Industry Scenario
  14. Conclusions

Figures (6)

  • Figure 1: (a1) An RGB image of a linear seam model; (a2) A point cloud of a linear seam model; (b1) An RGB image of a curved seam model; (b2) A point cloud of a curved seam model.
  • Figure 2: Overview of Coarse-to-Fine Detection of Multiple Seams via Edge Extraction for Robotic Welding
  • Figure 3: Results in different processes for three workpieces. (a) RGB Images; (b) Depth Maps; (c) Raw Point Cloud; (d) Segmentation Results; (e) The red area represents the point cloud cropped according to the segmentation information; (f) The light green area represents the extracted weld edges point cloud; (g) Robot-executable Welding Path with 6-DOF.
  • Figure 4: A robotic welding system consists of a UR5 robotic arm, an RGB-D camera, a 3D-printed welding torch, and a workpiece to be welded.
  • Figure 5: The changes in the quantity of downsampled point clouds and the corresponding variations in the maximum error and root mean square error (RMSE) resulting from selecting different grid sizes.
  • ...and 1 more figures