CoBOS: Constraint-Based Online Scheduler for Human-Robot Collaboration

TL;DR

CoBOS tackles the challenge of coordinating humans and robots under uncertainty by introducing a constraint-based online scheduling approach that continually reschedules in response to new observations. It integrates Constraint Programming with Behavior Trees in an online planning–acting loop, using a three-phase task model, NoOverlap constraints for shared resources, and real-time model updates to handle delays and human rejections. In a large probabilistic simulation study (56,000 runs) CoBOS consistently outperformed greedy and dynamicAllocation baselines, with especially large gains on more complex task classes, and demonstrated promising signs in initial real-robot experiments. The work also provides a probabilistic simulation framework and publicly available code/data to support further research in uncertainty-aware HRC scheduling and decision-making.

Abstract

Assembly processes involving humans and robots are challenging scenarios because the individual activities and access to shared workspace have to be coordinated. Fixed robot programs leave no room to diverge from a fixed protocol. Working on such a process can be stressful for the user and lead to ineffective behavior or failure. We propose a novel approach of online constraint-based scheduling in a reactive execution control framework facilitating behavior trees called CoBOS. This allows the robot to adapt to uncertain events such as delayed activity completions and activity selection (by the human). The user will experience less stress as the robotic coworkers adapt their behavior to best complement the human-selected activities to complete the common task. In addition to the improved working conditions, our algorithm leads to increased efficiency, even in highly uncertain scenarios. We evaluate our algorithm using a probabilistic simulation study with 56000 experiments. We outperform all other compared methods by a margin of 4-10%. Initial real robot experiments using a Franka Emika Panda robot and human tracking based on HTC Vive VR gloves look promising.

Figures (8)

• Figure 1: CoBOS is a constraint-based online scheduler for human-robot collaboration. It sequences tasks and allocates them to multiple actors based on uncertain information and incorporates observations during execution via event-driven rescheduling.
• Figure 2: Task graph of each sub-task consisting of the three phases preparation, execution, and completion. An optional waiting phase is allowed between preparation and execution to accommodate for an occupied shared area or unmet task dependencies.
• Figure 3: The final state of the shared assembly task. The precedence graph is shown on the right.
• Figure 4: Our proposed probabilistic closed-loop multi-actor simulation is a test environment template for central multi-actor coordination agents. The agent, for example CoBOS, requests, based on a job description and continuous observations, actions from the actors. The actors, e.g., a human worker or a robot, act upon these according to their policy. Humans, for example, might reject a task, while robots generally will accept assigned tasks. The environment decides for values of uncontrollable variables and compiles the observation message at each time step. All components are deterministic for a given random seed.
• Figure 5: Graphical User Interface for the real setup. A dynamic Gantt chart (top-left) shows the current state and schedule of the task. The dependency graph (bottom-left) shows the task-specific dependencies. On the right side, the current task details are displayed for the robot and the user. Human feedback about task completions or task rejections is collected via buttons.