Cognitive Load-based Affective Workload Allocation for Multi-human Multi-robot Teams

TL;DR

The results demonstrate the performance and effectiveness of the proposed controller and highlight the importance of incorporating both subjective and objective measurements of the operators' cognitive workload as well as seeking consent for workload transitions, to enhance the performance of multi-human multi-robot teams.

Abstract

The interaction and collaboration between humans and multiple robots represent a novel field of research known as human multi-robot systems. Adequately designed systems within this field allow teams composed of both humans and robots to work together effectively on tasks such as monitoring, exploration, and search and rescue operations. This paper presents a deep reinforcement learning-based affective workload allocation controller specifically for multi-human multi-robot teams. The proposed controller can dynamically reallocate workloads based on the performance of the operators during collaborative missions with multi-robot systems. The operators' performances are evaluated through the scores of a self-reported questionnaire (i.e., subjective measurement) and the results of a deep learning-based cognitive workload prediction algorithm that uses physiological and behavioral data (i.e., objective measurement). To evaluate the effectiveness of the proposed controller, we use a multi-human multi-robot CCTV monitoring task as an example and carry out comprehensive real-world experiments with 32 human subjects for both quantitative measurement and qualitative analysis. Our results demonstrate the performance and effectiveness of the proposed controller and highlight the importance of incorporating both subjective and objective measurements of the operators' cognitive workload as well as seeking consent for workload transitions, to enhance the performance of multi-human multi-robot teams.

Figures (16)

• Figure 1: Conceptual illustration of the deep reinforcement learning (DRL)-based affective workload allocation controller (AWAC) for multi-human multi-robot (MH-MR) teams. More details can be found at Session \ref{['sec:proposed_system']} or supplementary website: https://sites.google.com/view/affective-workload-allocation.
• Figure 2: A learning diagram of the proposed DRL model.
• Figure 3: Illustration of (a) MRS testbed for conducting surveillance missions and (b) Participants' workspace and wearable biosensors (red) and behavioral-monitoring devices (blue) used to collect physiological features and behavioral data.
• Figure 4: The experiment protocol used in the team-based user experiment involves three phases: Baseline, Main, and Evaluation phases. In the Main phase, a set is repeated three times for the CCTV monitoring task. This procedure is repeated eight times with different workload allocation methods randomly selected from Task A to H.
• Figure 5: An illustration of the HPM calculated using the ISA score and predicted cognitive workload.