Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Joint Prediction of Human Motions and Actions in Human-Robot Collaboration

Alessandra Bulanti, Alessandro Carfì, Fulvio Mastrogiovanni

Abstract

Fluent human--robot collaboration requires robots to continuously estimate human behaviour and anticipate future intentions. This entails reasoning jointly about \emph{continuous movements} and \emph{discrete actions}, which are still largely modelled in isolation. In this paper, we introduce \textsf{MA-HERP}, a hierarchical and recursive probabilistic framework for the \emph{joint estimation and prediction} of human movements and actions. The model combines: (i) a hierarchical representation in which movements compose into actions through admissible Allen interval relations, (ii) a unified probabilistic factorisation coupling continuous dynamics, discrete labels, and durations, and (iii) a recursive inference scheme inspired by Bayesian filtering, alternating top-down action prediction with bottom-up sensory evidence. We present a preliminary experimental evaluation based on neural models trained on musculoskeletal simulations of reaching movements, showing accurate motion prediction, robust action inference under noise, and computational performance compatible with on-line human--robot collaboration.

Joint Prediction of Human Motions and Actions in Human-Robot Collaboration

Abstract

Fluent human--robot collaboration requires robots to continuously estimate human behaviour and anticipate future intentions. This entails reasoning jointly about \emph{continuous movements} and \emph{discrete actions}, which are still largely modelled in isolation. In this paper, we introduce \textsf{MA-HERP}, a hierarchical and recursive probabilistic framework for the \emph{joint estimation and prediction} of human movements and actions. The model combines: (i) a hierarchical representation in which movements compose into actions through admissible Allen interval relations, (ii) a unified probabilistic factorisation coupling continuous dynamics, discrete labels, and durations, and (iii) a recursive inference scheme inspired by Bayesian filtering, alternating top-down action prediction with bottom-up sensory evidence. We present a preliminary experimental evaluation based on neural models trained on musculoskeletal simulations of reaching movements, showing accurate motion prediction, robust action inference under noise, and computational performance compatible with on-line human--robot collaboration.

Paper Structure

This paper contains 5 sections, 11 equations, 6 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Hierarchical Representation of Movements and Actions
  3. Recursive Estimation and Prediction on a Joint Movement-Action Model
  4. Experimental Evaluation
  5. Conclusions

Figures (6)

  • Figure 1: An illustrative example of a coffee-making scenario in which the prediction of human motions and actions can improve the perceived collaboration fluency to a great extent.
  • Figure 2: Hierarchical structure of movements and actions MA-HERP. Level $h=0$ encodes continuous movement segments, while levels $h\ge1$ encode discrete actions. Edges denote temporal relations defined via Allen's interval algebra.
  • Figure 3: A musculoskeletal model executing a reaching movement. Three poses are shown from left to right.
  • Figure 4: Predictions (red) in configuration space for $\ell_1 = \textsf{I}\to\textsf{A}$ obtained with (\ref{['fig:prediction']}) the model trained on D-0 and (\ref{['fig:prediction30']}) the model trained on D-0-10-30, compared against ground truth (black).
  • Figure 5: Confusion matrices for D-0 (top row) and D-0-10-30 (bottom row) datasets. P denotes precision, R the recall, and F1 the Macro F1 score.
  • ...and 1 more figures

Theorems & Definitions (6)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Definition 5
  • Definition 6