A Virtual Fencing Framework for Safe and Efficient Collaborative Robotics

TL;DR

The paper tackles real-time safety in human-robot collaboration by proposing a modular virtual fencing framework that integrates RGB-based human detection with zone-aware control and constrained optimization. Safety is defined as $\mathcal{X}_{\mathrm{s}}=\bigcap_{i\in \mathscr{M}}\mathcal{X}_{\mathrm{s}}^i$ with data-driven, ambient-camera constraints, while a zone-based perception map assigns detections to motion commands across a central critical zone and peripheral increased-attention zones. Motion is governed by $J(d)=\alpha (d-d_{\text{desired}})^2+\beta (d-d_{\text{prev}})^2$ subject to $d_{\min} \le d \le d_{\max}$, solved with Sequential Quadratic Programming (SQP) to ensure smooth, risk-aware speed adjustments in real time. Experiments on a UR16e setup with YOLOv8n/ONNXRuntime demonstrate improved operational efficiency (OE ~66%), modest latency (~33 ms), and high collision-avoidance rates (~98%), validating the approach’s practicality for dynamic industrial environments.

Abstract

Collaborative robots (cobots) increasingly operate alongside humans, demanding robust real-time safeguarding. Current safety standards (e.g., ISO 10218, ANSI/RIA 15.06, ISO/TS 15066) require risk assessments but offer limited guidance for real-time responses. We propose a virtual fencing approach that detects and predicts human motion, ensuring safe cobot operation. Safety and performance tradeoffs are modeled as an optimization problem and solved via sequential quadratic programming. Experimental validation shows that our method minimizes operational pauses while maintaining safety, providing a modular solution for human-robot collaboration.

Figures (4)

• Figure 1: Demonstration of a collaborative robot workspace with camera connected to embedded computer to detect human and notify the robot with safeguarding techniques. White dashed lines represent camera field of view.
• Figure 2: The robot operates at normal speed when no person is detected (top left). However, when a person is detected (highlighted in yellow) in the increased attention zone (top right), it slows down. When a person is detected (highlighted in red) in the critical zone (bottom right), the robot stops. Once the person moves back to the increased attention zone (bottom left), the robot resumes slow movement. Once the area is clear, it returns to normal speed.
• Figure 3: Experimental setup featuring a UR16e cobot, integrated with an Arducam IMX477 camera and an embedded AI computational unit.
• Figure 4: The effectiveness of velocity smoothening with and without SQP-based optimization. Zone-based detection is employed in both cases.