neuROSym: Deployment and Evaluation of a ROS-based Neuro-Symbolic Model for Human Motion Prediction

TL;DR

This work tackles the challenge of trustworthy, online human motion prediction for mobile robots operating in human environments. It introduces neuROSym, a ROS-based package that deploys and visualizes neural-only and neuro-symbolic predictors (SGAN and NeuroSyM) in real time, leveraging Qualitative Trajectory Calculus (QTC) to encode spatial interactions. Through hardware experiments on a TIAGo robot across two motion-pattern scenarios, NeuroSyM demonstrates lower ADE and FDE than SGAN, indicating improved prediction accuracy with a modest increase in runtime. The package provides a practical tool for real-time evaluation and underscores the benefits of neuro-symbolic context integration for safer, more reliable human-robot interaction in real-world settings.

Abstract

Autonomous mobile robots can rely on several human motion detection and prediction systems for safe and efficient navigation in human environments, but the underline model architectures can have different impacts on the trustworthiness of the robot in the real world. Among existing solutions for context-aware human motion prediction, some approaches have shown the benefit of integrating symbolic knowledge with state-of-the-art neural networks. In particular, a recent neuro-symbolic architecture (NeuroSyM) has successfully embedded context with a Qualitative Trajectory Calculus (QTC) for spatial interactions representation. This work achieved better performance than neural-only baseline architectures on offline datasets. In this paper, we extend the original architecture to provide neuROSym, a ROS package for robot deployment in real-world scenarios, which can run, visualise, and evaluate previous neural-only and neuro-symbolic models for motion prediction online. We evaluated these models, NeuroSyM and a baseline SGAN, on a TIAGo robot in two scenarios with different human motion patterns. We assessed accuracy and runtime performance of the prediction models, showing a general improvement in case our neuro-symbolic architecture is used. We make the neuROSym package1 publicly available to the robotics community.

Figures (6)

• Figure 1: Deployment of neuROSym for online and context-aware human motion prediction, with real-time visualisation. The three blocks are ROS nodes, while the filled arrows and the dashed ones represent the online and offline inference, respectively. Each arrows label indicates the type of messages published and/or subscribed to by each node.
• Figure 2: The NeuroSyM pooling mechanism with prior QTC knowledge injected into the output of the relative pose embedding layer.
• Figure 3: An example of QTC$_{C1}$ representation of interactions between three body points $P_h$, $P_r$, and $P_s$.
• Figure 4: (a) Experimental Scenario A with two humans walking parallel to each other towards their goal (room end) and back, repetitively. The online trajectory prediction is performed by models trained on the UCY-Zara01 dataset. (b) Experimental Scenario B with two humans crossing each other's path. Here the models are trained on the THOR dataset. (c) RViz visualization of the Bayes People Tracker with human bounding-boxes extracted from the robot's LiDAR point-clouds.
• Figure 5: (Top) Full human motion trajectories in Scenario B, where two people $H_1$ and $H_2$ (dynamic objects) move back and forth to their destinations (static objects), crossing each other's path and avoiding collisions. (Bottom) Snapshots at frames t = 8, 10, and 16s, from left to right, respectively.