Multi-Source Human-in-the-Loop Digital Twin Testbed for Connected and Autonomous Vehicles in Mixed Traffic Flow

Abstract

In the emerging mixed traffic environments, Connected and Autonomous Vehicles (CAVs) have to interact with surrounding human-driven vehicles (HDVs). This paper introduces MSH-MCCT (Multi-Source Human-in-the-Loop Mixed Cloud Control Testbed), a novel CAV testbed that captures complex interactions between various CAVs and HDVs. Utilizing the Mixed Digital Twin concept, which combines Mixed Reality with Digital Twin, MSH-MCCT integrates physical, virtual, and mixed platforms, along with multi-source control inputs. Bridged by the mixed platform, MSH-MCCT allows human drivers and CAV algorithms to operate both physical and virtual vehicles within multiple fields of view. Particularly, this testbed facilitates the coexistence and real-time interaction of physical and virtual CAVs \& HDVs, significantly enhancing the experimental flexibility and scalability. Experiments on vehicle platooning in mixed traffic showcase the potential of MSH-MCCT to conduct CAV testing with multi-source real human drivers in the loop through driving simulators of diverse fidelity. The videos for the experiments are available at our project website: https://dongjh20.github.io/MSH-MCCT.

Figures (12)

• Figure 1: Schematics for classical DT and mixedDT. (a) In classical DT, the virtual space is typically a digital replica of the physical space. (b) In mixedDT, the virtual space and the physical space are integrated into the mixed space, where physical and virtual entities could coexist and interact with each other.
• Figure 2: Schematic of MSH-MCCT. In the mixed platform, the physical vehicles in the physical platform (outlined in gold), the virtual vehicles in the virtual platform (outlined in cyan) and the virtual vehicle in the driving environment of a driving simulator (outlined in red) coexist and interact with each other in real-time. In contrast, in existing platforms zayas2021digitalzayas2021ids, physical and virtual vehicles operate exclusively within their respective environments, without effective integration or interactive mechanisms established between them. The multi-source control inputs includes human drivers via various-fidelity levels of driving simulators and diverse CAV algorithms. The visualization of the mixed platform shown in (c) is provided by a MR device. The demonstration videos are available at our project website: https://dongjh20.github.io/MSH-MCCT.
• Figure 3: The methodological framework for conducting CAV testing with multi-source human drivers in the loop via driving simulators in MSH-MCCT. Both physical and virtual vehicles could be controlled by human drivers and CAV algorithms through a cloud-based intermediary. Consequently, the four types of vehicles, i.e., physical HDVs, virtual HDVs, physical CAVs and virtual CAVs, coexist and interact with each other simultaneously within the mixed platform.
• Figure 4: Driving field-of-view and prompt panel. In (a) and (b), the snapshots of human-in-the-loop experiments based on physical and virtual field-of-view are presented respectively; see the screens for the specific view. In (c), the real-time speed prompt of the preceding vehicle in the virtual driving field-of-view is displayed on the white panel on the top of the preceding vehicle.
• Figure 5: Control modes diagram. Human drivers and CAV algorithms could control both the physical vehicles and the virtual vehicles via a cloud-based intermediary.

Theorems & Definitions (3)

• Remark 1: A novel paradigm for human-in-the-loop driving simulations
• Remark 2: "Hot-swapping" capability in MSH-MCCT
• Remark 3: Integration of diverse platforms and environments