Integrating Uncertainty-Aware Human Motion Prediction into Graph-Based Manipulator Motion Planning

TL;DR

This work addresses safe, proactive human–robot collaboration in disassembly by integrating uncertainty‑aware human motion prediction with a graph‑based neural motion planner. An LSTM predictor encodes human arm bone poses and uses Monte Carlo dropout (MCDS) to quantify predictive uncertainty, producing a distribution of likely future poses. These uncertainty representations are embedded as nodes/edges in a workspace graph that a GNN uses to generate near‑optimal, collision‑free robot motions, with bi‑directional planning to reduce error accumulation. Experimental results on a 6‑DOF manipulator demonstrate improved safety and smoother end‑effector trajectories when predictions are incorporated, along with a tunable trade‑off between inference time and uncertainty diversity. The framework offers a scalable, data‑driven approach to proactive HRC, applicable to dynamic industrial tasks where human actions are variable and safety margins must adapt in real time.

Abstract

There has been a growing utilization of industrial robots as complementary collaborators for human workers in re-manufacturing sites. Such a human-robot collaboration (HRC) aims to assist human workers in improving the flexibility and efficiency of labor-intensive tasks. In this paper, we propose a human-aware motion planning framework for HRC to effectively compute collision-free motions for manipulators when conducting collaborative tasks with humans. We employ a neural human motion prediction model to enable proactive planning for manipulators. Particularly, rather than blindly trusting and utilizing predicted human trajectories in the manipulator planning, we quantify uncertainties of the neural prediction model to further ensure human safety. Moreover, we integrate the uncertainty-aware prediction into a graph that captures key workspace elements and illustrates their interconnections. Then a graph neural network is leveraged to operate on the constructed graph. Consequently, robot motion planning considers both the dependencies among all the elements in the workspace and the potential influence of future movements of human workers. We experimentally validate the proposed planning framework using a 6-degree-of-freedom manipulator in a shared workspace where a human is performing disassembling tasks. The results demonstrate the benefits of our approach in terms of improving the smoothness and safety of HRC. A brief video introduction of this work is available as the supplemental materials.

Figures (8)

• Figure 1: Overview of the proposed HRC motion planning framework: 1) Observed human motions are used to compute future uncertainty-aware human motions; 2) The collaboration workspace is converted to a graph. It preserves the structural attributes of objects by employing multiple nodes and edges. The key elements and characteristics of the workspace are depicted through nodes with features, and their connections are established through edges. Note that we only show a few of the nodes in the figure to simplify the illustration. 3) The uncertainty-aware prediction is also represented using nodes and edges and naturally integrated into the overall graph. 4) The GNN-based motion planner eventually generates a safe robot configuration, directing the robot toward the desired goal, while avoiding the moving human agent in close vicinity.
• Figure 2: The uncertainty quantification of the prediction model: green dots are the observed human joints, red dots are the predicted human joints, and the uncertainty-aware prediction is generated based on the predictive distribution and includes multiple possible human arm poses at each time step.
• Figure 3: The representation of objects in the workspace and the simplified illustration of the overall graph: the uncertainty-aware prediction is represented using red color. We use dots A, B, and C to simplify the graph representation. Blue dot A indicates the robot's current state, purple dot B denotes the robot's goal state, and green dot C indicates the obstacle's state. All dots contain multiple nodes and edges based on their own structural attributes.
• Figure 4: The process of motion generation: red dot D indicates the predicted human arm after MCDS, multiple GNN blocks are used to update the embeddings of nodes, and GNN finally outputs the robot configuration of the next step.
• Figure 5: The experimental platform: a, b, and c represent three locations in the table, tool box, and desktop, respectively. Human motion A demonstrates a scenario in which a human worker initially disassembles a hard disk on the table, then reaches towards the tool box to get a new screwdriver, and finally resumes the disassembly task. Human motion B demonstrates a scenario where a human worker initially disassembles a hard disk on the table, then grabs a component from the disassembled desktop, and eventually puts the component on the table.