Analysis of Deep-Learning Methods in an ISO/TS 15066-Compliant Human-Robot Safety Framework

TL;DR

This work tackles the efficiency bottleneck of ISO/TS 15066 compliant HRC by introducing a DL-based HRSF that dynamically tunes robot speeds according to body-part–specific separation distances, enabled by RGB-D perception. It systematically benchmarks four DL approaches across body and body-part localization, maps their outputs to per-part safety thresholds, and derives safety quantities such as the minimum separation distance $S_p$ and per-part velocities $\dot{z}_{Max}$. On a KUKA iiwa, the framework demonstrates cycle-time reductions up to $35\%$ versus no safety and more than $15\%$ against a laser-scanner baseline, with body-part segmentation delivering the strongest gains and robustness. However, the study remains a feasibility assessment (single subject, limited repetitions) and calls for broader validation, multi-camera setups, and formal safety certification before production deployment.

Abstract

Over the last years collaborative robots have gained great success in manufacturing applications where human and robot work together in close proximity. However, current ISO/TS-15066-compliant implementations often limit the efficiency of collaborative tasks due to conservative speed restrictions. For this reason, this paper introduces a deep-learning-based human-robot-safety framework (HRSF) that aims at a dynamical adaptation of robot velocities depending on the separation distance between human and robot while respecting maximum biomechanical force and pressure limits. The applicability of the framework was investigated for four different deep learning approaches that can be used for human body extraction: human body recognition, human body segmentation, human pose estimation, and human body part segmentation. Unlike conventional industrial safety systems, the proposed HRSF differentiates individual human body parts from other objects, enabling optimized robot process execution. Experiments demonstrated a quantitative reduction in cycle time of up to 15% compared to conventional safety technology.

Figures (12)

• Figure S1: Visualized results of the applied deep learning techniques for (a) human body recognition SSD_used and (b) human body segmentation Mask_RCNN in color image (left) and depth image (right).
• Figure S2: Visualized results of the applied deep learning techniques for (a) human pose estimation DeepPose and (b) human body part segmentation Human_body_part_parsing in color image (left) and depth image (right).
• Figure S3: Individual latency contributions determined for each of the analyzed algorithms within the HRSF.
• Figure S4: Minimal separation distance determination according to the current robot pose via the application of the cuboid-shaped robot protective hull approach for (a) human-body-related and (b) human-body-part-related approaches.
• Figure S5: (a) Experimental setup for data acquisition. (b) Position of markers and indications of whether a marker is used for human pose estimation, human body part segmentation ,or only for human-body-related accuracy validation. All of the markers chosen for human pose estimation and human body part segmentation were also considered for human-body-related measurements.