Online Action Recognition for Human Risk Prediction with Anticipated Haptic Alert via Wearables

TL;DR

The work tackles the challenge of real-time biomechanical risk assessment during manual lifting, proposing an online framework that fuses wearable sensing, inverse kinematics/dynamics-based state estimation, and a Guided Mixture of Experts for simultaneous action recognition and motion prediction. By applying the Revised NIOSH Lifting Equation over predicted futures, the system provides anticipatory haptic alerts to workers, enabling proactive risk mitigation. The contributions include adapting GMoE for lifting tasks, online RNLE-based risk computation, and validation with the iFeel wearable in laboratory lifts, achieving close-to-real-time performance with action accuracies around 0.89–0.93 and meaningful motion forecasts. This approach advances workplace safety by enabling continuous, proactive ergonomic feedback in unstructured environments using wearable sensors and learning-based prediction.

Abstract

This paper proposes a framework that combines online human state estimation, action recognition and motion prediction to enable early assessment and prevention of worker biomechanical risk during lifting tasks. The framework leverages the NIOSH index to perform online risk assessment, thus fitting real-time applications. In particular, the human state is retrieved via inverse kinematics/dynamics algorithms from wearable sensor data. Human action recognition and motion prediction are achieved by implementing an LSTM-based Guided Mixture of Experts architecture, which is trained offline and inferred online. With the recognized actions, a single lifting activity is divided into a series of continuous movements and the Revised NIOSH Lifting Equation can be applied for risk assessment. Moreover, the predicted motions enable anticipation of future risks. A haptic actuator, embedded in the wearable system, can alert the subject of potential risk, acting as an active prevention device. The performance of the proposed framework is validated by executing real lifting tasks, while the subject is equipped with the iFeel wearable system.

Figures (7)

• Figure 1: An overview of the proposed framework.
• Figure 2: Data flow of the proposed framework, composed of an online and offline phase.
• Figure 3: The three phases composing the lifting activity.
• Figure 4: Adapted structure of Guided Mixture of Experts architecture for action recognition and motion prediction.
• Figure 5: Confusion matrix for the action classification.