Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Precise Workcell Sketching from Point Clouds Using an AR Toolbox

Krzysztof Zieliński, Bruce Blumberg, Mikkel Baun Kjærgaard

TL;DR

This work proposes a novel solution for 3D workcell sketching from point clouds that allows users to refine raw point clouds using an Augmented Reality (AR) interface that leverages their knowledge and the real-world 3D environment.

Abstract

Capturing real-world 3D spaces as point clouds is efficient and descriptive, but it comes with sensor errors and lacks object parametrization. These limitations render point clouds unsuitable for various real-world applications, such as robot programming, without extensive post-processing (e.g., outlier removal, semantic segmentation). On the other hand, CAD modeling provides high-quality, parametric representations of 3D space with embedded semantic data, but requires manual component creation that is time-consuming and costly. To address these challenges, we propose a novel solution that combines the strengths of both approaches. Our method for 3D workcell sketching from point clouds allows users to refine raw point clouds using an Augmented Reality (AR) interface that leverages their knowledge and the real-world 3D environment. By utilizing a toolbox and an AR-enabled pointing device, users can enhance point cloud accuracy based on the device's position in 3D space. We validate our approach by comparing it with ground truth models, demonstrating that it achieves a mean error within 1cm - significant improvement over standard LiDAR scanner apps.

Precise Workcell Sketching from Point Clouds Using an AR Toolbox

TL;DR

This work proposes a novel solution for 3D workcell sketching from point clouds that allows users to refine raw point clouds using an Augmented Reality (AR) interface that leverages their knowledge and the real-world 3D environment.

Abstract

Capturing real-world 3D spaces as point clouds is efficient and descriptive, but it comes with sensor errors and lacks object parametrization. These limitations render point clouds unsuitable for various real-world applications, such as robot programming, without extensive post-processing (e.g., outlier removal, semantic segmentation). On the other hand, CAD modeling provides high-quality, parametric representations of 3D space with embedded semantic data, but requires manual component creation that is time-consuming and costly. To address these challenges, we propose a novel solution that combines the strengths of both approaches. Our method for 3D workcell sketching from point clouds allows users to refine raw point clouds using an Augmented Reality (AR) interface that leverages their knowledge and the real-world 3D environment. By utilizing a toolbox and an AR-enabled pointing device, users can enhance point cloud accuracy based on the device's position in 3D space. We validate our approach by comparing it with ground truth models, demonstrating that it achieves a mean error within 1cm - significant improvement over standard LiDAR scanner apps.
Paper Structure (17 sections, 5 figures, 1 table)

This paper contains 17 sections, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related work
  3. Point cloud processing in AR and VR
  4. 3D sketching in AR
  5. Point cloud processing methods
  6. Method
  7. Point cloud collection flow
  8. Point cloud processing tools
  9. Evaluation and Results
  10. Evaluation scenarios
  11. Proposed method implementation
  12. Evaluation setup
  13. Evaluation metrics
  14. Evaluation metrics results
  15. Examples of robot automation workcells
  16. ...and 2 more sections

Figures (5)

  • Figure 1: Digital representation types placed based on complexity and accuracy levels. Top: representation characteristics, Bottom: robot applications.
  • Figure 2: The flow of the point cloud collection with each tool available in the toolbox: a) outlier removal, b) downsampling, c) point creation using primitives, d) eraser sponge, and e) eraser spray.
  • Figure 3: 3D models that were used to obtain evaluation metrics in \ref{['tab:error_3DModels']}. Objects sorted based on test scenarios: \ref{['sce:1']} shiny surface difficult to model --- (a,b,f); \ref{['sce:2']} scans containing unnecessary data --- (e); \ref{['sce:3']} scans containing background noise, e.g., floors --- (a--f); \ref{['sce:4']} --- objects missing geometrical data at the time of the scan (d).
  • Figure 4: Cloud/mesh distance calculation.
  • Figure 5: Scans of 4 robot automation workcells utilizing our proposed method: \ref{['workcell:1']}) material handling, \ref{['workcell:2']}) machine tending, \ref{['workcell:3']}) palletizing, \ref{['workcell:4']}) assembly. The last row describes the used tools. The arrows present visible changes.