PRO-MIND: Proximity and Reactivity Optimisation of robot Motion to tune safety limits, human stress, and productivity in INDustrial settings

TL;DR

PRO-MIND tackles the safety–comfort–productivity trade-off in industrial human–robot collaboration by online integrating human attention and psycho-physical state into trajectory planning. It combines online camera-based monitoring of attention and mental effort with multi-objective time/jerk optimization on quintic $B$-splines, adaptively resizing physical and cognitive safety zones and locally modifying the path to preserve smoothness. A Pareto-front approach selects timing solutions that balance speed and smoothness, guided by HRV metrics and camera cues, yielding personalized trajectories with reduced psycho-physical stress and improved collaboration fluency across two realistic tasks. Results show safer proximity maintenance, reduced idle time, and competitive productivity compared to state-of-the-art baselines, supporting the framework’s potential for scalable, worker-centric automation in industry.

Abstract

Despite impressive advancements of industrial collaborative robots, their potential remains largely untapped due to the difficulty in balancing human safety and comfort with fast production constraints. To help address this challenge, we present PRO-MIND, a novel human-in-the-loop framework that leverages valuable data about the human co-worker to optimise robot trajectories. By estimating human attention and mental effort, our method dynamically adjusts safety zones and enables on-the-fly alterations of the robot path to enhance human comfort and optimal stopping conditions. Moreover, we formulate a multi-objective optimisation to adapt the robot's trajectory execution time and smoothness based on the current human psycho-physical stress, estimated from heart rate variability and frantic movements. These adaptations exploit the properties of B-spline curves to preserve continuity and smoothness, which are crucial factors in improving motion predictability and comfort. Evaluation in two realistic case studies showcases the framework's ability to restrain the operators' workload and stress and to ensure their safety while enhancing human-robot productivity. Further strengths of PRO-MIND include its adaptability to each individual's specific needs and sensitivity to variations in attention, mental effort, and stress during task execution.

Figures (15)

• Figure 1: PRO-MIND framework adapting CoBot trajectory to promote human well-being and system productivity. Note that computational blue modules refer to human monitoring, red modules determine the path and timing adaptation of the CoBot trajectory, and variables in grey are provided offline.
• Figure 2: $x$, $y$, and $z$ coordinates of end-effector trajectory defined by a B-spline curve passing through a set of waypoints (black dots) and two virtual points (grey dots). Time intervals $h_l$ between consecutive points, defining optimisation vector $\bf{h}$, are highlighted.
• Figure 3: Conceptual illustration exemplifying the dynamic scaling of physical and cognitive-grounded safety zones based on human awareness.
• Figure 4: 3D-view of end-effector trajectory highlighting control points and their modification while the trajectory is accomplished.
• Figure 5: Pareto Optimal Front $\mathcal{F}$ (gray circles, normalised for sake of clarity) resulting by minimum time-jerk optimisation. Blue dots denote the front $\mathcal{F}^*$ after the downsampling procedure.