ContactRL: Safe Reinforcement Learning based Motion Planning for Contact based Human Robot Collaboration

TL;DR

This work tackles safe physical human-robot collaboration by enabling a robot to learn contact-aware motion through reinforcement learning, integrating force-feedback in the reward to reduce harmful contact. A kinetic-energy based Control Barrier Function (eCBF) shield is deployed at run-time to guarantee safety without retraining, linking learned policy execution to strict safety bounds. In simulation, ContactRL achieves high task success with minimal safety violations and outperforms state-of-the-art constrained RL baselines, while real-world experiments confirm sub-10 N contact forces across diverse participants and objects. The approach decouples safety guarantees from performance, enabling robust, fluid handovers and extending to other contact-rich tasks in assistive and industrial settings.

Abstract

In collaborative human-robot tasks, safety requires not only avoiding collisions but also ensuring safe, intentional physical contact. We present ContactRL, a reinforcement learning (RL) based framework that directly incorporates contact safety into the reward function through force feedback. This enables a robot to learn adaptive motion profiles that minimize human-robot contact forces while maintaining task efficiency. In simulation, ContactRL achieves a low safety violation rate of 0.2\% with a high task success rate of 87.7\%, outperforming state-of-the-art constrained RL baselines. In order to guarantee deployment safety, we augment the learned policy with a kinetic energy based Control Barrier Function (eCBF) shield. Real-world experiments on an UR3e robotic platform performing small object handovers from a human hand across 360 trials confirm safe contact, with measured normal forces consistently below 10N. These results demonstrate that ContactRL enables safe and efficient physical collaboration, thereby advancing the deployment of collaborative robots in contact-rich tasks.

Figures (7)

• Figure 1: Positional-deviation profile $\mathbf d(t)$ from start $t_0$ to finish $t_N$. We seek a trajectory minimizing task completion time that satisfies the terminal safety constraint $F_N(\mathbf d(t_N))\le F_\tau$, where $F_N$ is the contact force on the hand and $F_\tau$ is the safe contact limit, to ensure human comfort and safety during small object handover.
• Figure 2: ContactRL: An RL-based motion planning framework for human–robot close-contact interaction. The robot learns an adaptive motion profile based on a human-robot contact model. A rigid-body constraint is applied between the robot and the object to enable lift, while the normal contact force exerted on the human hand, ($F_N$) is feedback to the safety component of the reward function.
• Figure 3: Simulated environment with UR3e robot, object, and a simplified hand model. The right panel illustrates the components of the contact force applied to the hand during interaction.
• Figure 4: Ablation study of ContactRL reward function variants (RF1–RF5) with component weightings specified in Table I, evaluated over 1000 simulated episodes. RF5 achieves strong performance with minimal safety violations, outperforming other variants.
• Figure 5: Kinetic energy (left) and end-effector speed (right) evaluated over 100 simulated episodes with stochastic action space. The eCBF shield results in smoother dissipation of kinetic energy and reduction of end-effector speed, enhancing stability and safety.