Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Using 3-D LiDAR Data for Safe Physical Human-Robot Interaction

Sarthak Arora, Karthik Subramanian, Odysseus Adamides, Ferat Sahin

TL;DR

This work addresses safe physical human-robot interaction on industrial shop floors using 3D LiDAR data. It proposes an end-to-end perception pipeline that fuses multiple LiDAR frames and trains a YOLOv9 detector, along with a Safety Speed-and-Separation Monitoring controller based on 3D geometry and Jacobian-based velocity scaling. The study demonstrates with 17 participants that a 3D LiDAR-based approach can yield accurate distance estimation, robust detection, and safe robot operation, while mapping channels to point-cloud ROIs to reduce search space. It also discusses practical limitations, such as holes in dark-clothing scenarios, and outlines concrete directions for improving robustness with higher-resolution sensors and enhanced learning strategies.

Abstract

This paper explores the use of 3D lidar in a physical Human-Robot Interaction (pHRI) scenario. To achieve the aforementioned, experiments were conducted to mimic a modern shop-floor environment. Data was collected from a pool of seventeen participants while performing pre-determined tasks in a shared workspace with the robot. To demonstrate an end-to-end case; a perception pipeline was developed that leverages reflectivity, signal, near-infrared, and point-cloud data from a 3-D lidar. This data is then used to perform safety based control whilst satisfying the speed and separation monitoring (SSM) criteria. In order to support the perception pipeline, a state-of-the-art object detection network was leveraged and fine-tuned by transfer learning. An analysis is provided along with results of the perception and the safety based controller. Additionally, this system is compared with the previous work.

Using 3-D LiDAR Data for Safe Physical Human-Robot Interaction

TL;DR

This work addresses safe physical human-robot interaction on industrial shop floors using 3D LiDAR data. It proposes an end-to-end perception pipeline that fuses multiple LiDAR frames and trains a YOLOv9 detector, along with a Safety Speed-and-Separation Monitoring controller based on 3D geometry and Jacobian-based velocity scaling. The study demonstrates with 17 participants that a 3D LiDAR-based approach can yield accurate distance estimation, robust detection, and safe robot operation, while mapping channels to point-cloud ROIs to reduce search space. It also discusses practical limitations, such as holes in dark-clothing scenarios, and outlines concrete directions for improving robustness with higher-resolution sensors and enhanced learning strategies.

Abstract

This paper explores the use of 3D lidar in a physical Human-Robot Interaction (pHRI) scenario. To achieve the aforementioned, experiments were conducted to mimic a modern shop-floor environment. Data was collected from a pool of seventeen participants while performing pre-determined tasks in a shared workspace with the robot. To demonstrate an end-to-end case; a perception pipeline was developed that leverages reflectivity, signal, near-infrared, and point-cloud data from a 3-D lidar. This data is then used to perform safety based control whilst satisfying the speed and separation monitoring (SSM) criteria. In order to support the perception pipeline, a state-of-the-art object detection network was leveraged and fine-tuned by transfer learning. An analysis is provided along with results of the perception and the safety based controller. Additionally, this system is compared with the previous work.
Paper Structure (14 sections, 2 equations, 14 figures, 1 table, 1 algorithm)

This paper contains 14 sections, 2 equations, 14 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Literature Survey
  3. Methodology
  4. Setup & Calibration
  5. Data Collection and Labeling
  6. Network Training and Inference
  7. Human point cloud extraction
  8. Speed and Separation Monitoring Algorithm
  9. Experiment
  10. Results and Discussion
  11. Dataset
  12. Quantitative Results
  13. Qualitative Results
  14. Conclusions & Future Work

Figures (14)

  • Figure 1: An image showing different stages of the experiment setup used in this work. In image "A", the layout of the robot workspace is shown along with the exteroceptive sensors used in the setup (encircled in red and white). In image "B", the test subject is wearing a motion capture body suit for acquiring minimum distance associated with the human and robot. In image "C", the participant is wearing a reflective vest and reflective hardhat.
  • Figure 2: Control schema showing the complete system, our communication is powered by Robot Operating System (ROS).
  • Figure 3: Clean samples of the reflectivity, signal, near-IR, and a depth-wise stacked image of the first three. The annotation is overlayed on the grayscale images in black and in green on colored images.
  • Figure 4: An image showing human shape geometry extraction using a point-cloud along with a bounding determined from a reflectivity image. Points in blue represent the human body, points in black are rejected as background.
  • Figure 5: Flow diagram of the experiment performed as prescribed by the authors in savur_physiological_2022
  • ...and 9 more figures