Intelligent Framework for Human-Robot Collaboration: Dynamic Ergonomics and Adaptive Decision-Making

TL;DR

The paper tackles safety and ergonomics in industrial human-robot collaboration by proposing a unified framework that fuses advanced visual perception, non-invasive real-time ergonomic monitoring, and adaptive Behaviour Tree–based decision-making. It combines YOLO11 for object detection, UKF for tracking, OpenPose for 3D pose, SlowOnly for action recognition, and OWAS for continuous ergonomic assessment, all feeding a BT-driven allocator of robot and human roles. Experimental results demonstrate strong perception performance (mAP up to 72.4% at mAP@50:95), high intention recognition accuracy (92.5%), and rapider interventions (0.07 s BT reaction) with a 56% improvement over benchmarks in critical metrics. The framework’s non-invasive, modular design enables real-time adaptation to dynamic industrial tasks, promising enhanced operator safety and productivity; future work includes real-world deployment, multi-operator/multi-robot scenarios, and cross-domain extension.

Abstract

The integration of collaborative robots into industrial environments has improved productivity, but has also highlighted significant challenges related to operator safety and ergonomics. This paper proposes an innovative framework that integrates advanced visual perception, continuous ergonomic monitoring, and adaptive Behaviour Tree decision-making to overcome the limitations of traditional methods that typically operate as isolated components. Our approach synthesizes deep learning models, advanced tracking algorithms, and dynamic ergonomic assessments into a modular, scalable, and adaptive system. Experimental validation demonstrates the framework's superiority over existing solutions across multiple dimensions: the visual perception module outperformed previous detection models with 72.4% mAP@50:95; the system achieved high accuracy in recognizing operator intentions (92.5%); it promptly classified ergonomic risks with minimal latency (0.57 seconds); and it dynamically managed robotic interventions with exceptionally responsive decision-making capabilities (0.07 seconds), representing a 56% improvement over benchmark systems. This comprehensive solution provides a robust platform for enhancing human-robot collaboration in industrial environments by prioritizing ergonomic safety, operational efficiency, and real-time adaptability.

Figures (8)

• Figure 1: Proposed HRC framework integrating object detection and tracking (YOLO11, Unscented Kalman Filter), 3D human pose tracking (OpenPose), action recognition (SlowOnly), OWAS-based ergonomic assessment, and a Behaviour Trees decision-making system. Solid arrows represent direct data flows between primary functional components, while dashed arrows indicate indirect or complementary information exchanges that support the main processing pipeline. The architecture facilitates non-invasive continuous monitoring, dynamic human-robot role allocation, and ergonomic-based interventions to enhance safety, ergonomics, and productivity in industrial environments.
• Figure 2: Overview of the proposed human-robot collaboration framework. The architecture integrates three input modules (3D Human Pose Tracking, Action Recognition, and Object Detection and Tracking) that feed into the OWAS ergonomic assessment system. The ergonomic evaluation classifies postures according to back, arms, legs, and load parameters, incorporating a temporal sliding window to track sustained postures. Based on this assessment, the Behaviour Tree manages dynamic role allocation between human operator and robot, prioritizing robot intervention when biomechanical fatigue is detected. This integrated approach ensures operational efficiency while preserving worker ergonomic safety.
• Figure 3: OWAS (Ovako Working Posture Analysis System) evaluation matrix. The left section displays the coding system for back (1-4), arms (1-3), legs (1-7), and load (1-3) postures. The right section presents the action category matrix with color-coded risk levels (green: no action required; yellow: corrective actions required in the near future; orange: corrective actions should be done as soon as possible; red: corrective actions for improvement required immediately) based on combinations of back, arms, legs, and load values.
• Figure 4: Hierarchical structure of the implemented Behaviour Tree for dynamic role allocation. The tree starts with a ROOT Fallback node and branches through a Main Sequence into conditional checks for robot position, human sequence completion, biomechanical fatigue, and parcel weight. Based on these evaluations, the Action Sequence Fallback node directs execution either to robot-led actions (via No Walk With Parcel Sequence) or human-led operations with robot support (via Walk With Parcel Sequence).
• Figure 5: Distribution of postures classified according to OWAS classes during the experiment. Most of the postures belong to OWAS class 2 (moderate risk), with a small percentage in OWAS 4 (high risk).