LiHRA: A LiDAR-Based HRI Dataset for Automated Risk Monitoring Methods

TL;DR

LiHRA addresses the lack of realistic, multi-modal datasets for automated risk monitoring in human-robot interaction by providing a LiDAR-based dataset that fuses 3D point clouds, human keypoints, and robot joint states across six industrial scenarios with safe and hazardous variants. The hardware-integrated pipeline ensures accurate extrinsic calibration and synchronized data capture, yielding 4,431 labeled frames at ~10 Hz. A risk-monitoring method aligned with ISO/TS 15066 is demonstrated, combining contextual cues and external force estimation to quantify hazard levels over time. The dataset and methodology enable development of both classical and AI-driven RM approaches for real-time risk assessment and adaptive safety in HRI workspaces, including precise collision severity evaluation.

Abstract

We present LiHRA, a novel dataset designed to facilitate the development of automated, learning-based, or classical risk monitoring (RM) methods for Human-Robot Interaction (HRI) scenarios. The growing prevalence of collaborative robots in industrial environments has increased the need for reliable safety systems. However, the lack of high-quality datasets that capture realistic human-robot interactions, including potentially dangerous events, slows development. LiHRA addresses this challenge by providing a comprehensive, multi-modal dataset combining 3D LiDAR point clouds, human body keypoints, and robot joint states, capturing the complete spatial and dynamic context of human-robot collaboration. This combination of modalities allows for precise tracking of human movement, robot actions, and environmental conditions, enabling accurate RM during collaborative tasks. The LiHRA dataset covers six representative HRI scenarios involving collaborative and coexistent tasks, object handovers, and surface polishing, with safe and hazardous versions of each scenario. In total, the data set includes 4,431 labeled point clouds recorded at 10 Hz, providing a rich resource for training and benchmarking classical and AI-driven RM algorithms. Finally, to demonstrate LiHRA's utility, we introduce an RM method that quantifies the risk level in each scenario over time. This method leverages contextual information, including robot states and the dynamic model of the robot. With its combination of high-resolution LiDAR data, precise human tracking, robot state data, and realistic collision events, LiHRA offers an essential foundation for future research into real-time RM and adaptive safety strategies in human-robot workspaces.

Figures (6)

• Figure 1: LiHRA includes labeled point clouds from both safe and hazardous scenarios. The top row depicts a coexisting scenario: a human safely observing the robot (left) and an inattentive human colliding with it (right). The bottom row shows a handover scenario: a safe item transfer (left) and the robot approaching an unaware human (right).
• Figure 2: Schematic setup of the hardware to record LiHRA.
• Figure 3: Soft- and hardware pipeline from the creation of the LiHRA dataset.
• Figure 4: Estimated external force magnitudes during the coexistence scenarios.
• Figure 5: Estimated external force magnitudes during the collaboration scenarios.