KG-Planner: Knowledge-Informed Graph Neural Planning for Collaborative Manipulators

TL;DR

KG-Planner introduces a knowledge‑informed graph neural planner that explicitly encodes workspace structure as a graph and uses a layer‑wise propagating GNN to generate near‑optimal, collision‑free motions for high‑DOF collaborative manipulators. By training against an oracle planner and employing online bi‑directional planning, the method achieves efficient, safe motion planning in static and dynamic environments with humans, while preserving object connectivity and planning knowledge. The approach is validated in extensive static and dynamic experiments, showing favorable path costs and planning times compared with classical planners, and demonstrated generalization to unseen workspaces and real‑time human‑in‑the‑loop scenarios. The results suggest strong practical potential for rapid, reliable planning in human‑robot collaboration tasks such as disassembly, with immediate applicability to real‑time replanning and safety guarantees.

Abstract

This paper presents a novel knowledge-informed graph neural planner (KG-Planner) to address the challenge of efficiently planning collision-free motions for robots in high-dimensional spaces, considering both static and dynamic environments involving humans. Unlike traditional motion planners that struggle with finding a balance between efficiency and optimality, the KG-Planner takes a different approach. Instead of relying solely on a neural network or imitating the motions of an oracle planner, our KG-Planner integrates explicit physical knowledge from the workspace. The integration of knowledge has two key aspects: (1) we present an approach to design a graph that can comprehensively model the workspace's compositional structure. The designed graph explicitly incorporates critical elements such as robot joints, obstacles, and their interconnections. This representation allows us to capture the intricate relationships between these elements. (2) We train a Graph Neural Network (GNN) that excels at generating nearly optimal robot motions. In particular, the GNN employs a layer-wise propagation rule to facilitate the exchange and update of information among workspace elements based on their connections. This propagation emphasizes the influence of these elements throughout the planning process. To validate the efficacy and efficiency of our KG-Planner, we conduct extensive experiments in both static and dynamic environments. These experiments include scenarios with and without human workers. The results of our approach are compared against existing methods, showcasing the superior performance of the KG-Planner. A short video introduction of this work is available (video link provided in the paper).

Figures (8)

• Figure 1: The illustration of the graph representation: (a) The robot's current and desired goal states are represented using blue and green colors, where the nodes indicate the robot joints and the edges indicate the robot links. The static obstacles as well as the human in the workspace are represented using pink color. The nodes of static obstacles indicate their corners, and the nodes of the human indicate the joints of the human arm. (b) The blue nodes denote the joints of the robot's current state. To simplify the graph representation here, we represent the robot's goal state with six small green nodes, and all obstacles with one pink nodes. The goal and obstacle nodes are connected with each current joint node since they have effects on the robot's motion generation.
• Figure 2: The overview of planning manipulator motion using the proposed KG-Planner: (1) the graph constructor converts the workspace information $X$ of the step $t$ to the feature and adjacency matrices that imply the objects' features and connections, respectively. The black dot in the adjacency matrix indicates the node of the row is directionally connected with the node of the column. (2) The motion generator takes the constructed graph $G$ of the step $t$ as input to generate the manipulator configuration $\hat{\theta}$ of the step $t{+}1$.
• Figure 3: The update of node embedding in the GNN model: the joint node 4 receives the planning knowledge from its neighbors, $c_{34}$ means the feature vector sent from the joint node 3 to the joint node 4, and the node embedding update happens on all nodes simultaneously.
• Figure 4: The illustration of KG-Planner-based bi-directional planning: The KG-Planner takes the initial and target configurations as inputs to provide forward and backward predictions simultaneously. Linearly virtual interpolations are generated between two predictions, which try to directly connect two planning branches. If there is a collision between the connected planning branches and the environment, the predictions become the new inputs of the KG-Planner. We do such bi-directional planning recursively until a collision-free path is found.
• Figure 5: Experimental planning results in different environments: given random start and target manipulator configurations in the selected four environments, our bi-directional KG-Planner plans near-optimal motions.