Human-Aided Trajectory Planning for Automated Vehicles through Teleoperation and Arbitration Graphs

TL;DR

Disengagements in automated driving limit full autonomy, motivating remote planning support. The paper introduces a Teleoperation behavior within an arbitration-graph framework to modify planner constraints at runtime without altering existing planning components, enabling trajectory generation beyond nominal ODDs. It formalizes integration, demonstrates two simulation scenarios where longitudinal or lateral constraints are adjusted to bypass obstacles, and discusses safety verification and real-world deployment considerations. Overall, the approach offers a scalable, human-in-the-loop path to extend the operational design domain of automated vehicles using modular, verifiable decision-making.

Abstract

Teleoperation enables remote human support of automated vehicles in scenarios where the automation is not able to find an appropriate solution. Remote assistance concepts, where operators provide discrete inputs to aid specific automation modules like planning, is gaining interest due to its reduced workload on the human remote operator and improved safety. However, these concepts are challenging to implement and maintain due to their deep integration and interaction with the automated driving system. In this paper, we propose a solution to facilitate the implementation of remote assistance concepts that intervene on planning level and extend the operational design domain of the vehicle at runtime. Using arbitration graphs, a modular decision-making framework, we integrate remote assistance into an existing automated driving system without modifying the original software components. Our simulative implementation demonstrates this approach in two use cases, allowing operators to adjust planner constraints and enable trajectory generation beyond nominal operational design domains.

Figures (8)

• Figure 1: An automated vehicle stops in front of an obstacle blocking the lane, unable to proceed without human intervention. In the presented approach, the vehicle automatically senses its need for teleoperation support through the usage of an arbitration graph for decision-making. A human remote operator provides guidance by modifying the planning constraints (red and green lines), enabling the vehicle to pass the obstacle. This scenario and results are presented in-depth in \ref{['sec:results']}.
• Figure 2: The arbitration graph used in this paper during active teleoperation. It is a simplified version of the arbitration graph introduced in orzechowskiVerhaltensentscheidungFuerAutomatisierte2023 extended by the Teleoperation behavior component. Highlighted in green are currently active nodes, grayed out nodes are currently not applicable.
• Figure 3: Overview of trajectory based teleoperation concepts. In trajectory guidance, the defines all aspects (i.e. curvature and velocity) of the trajectory the shall execute. In waypoint guidance, the inputs waypoints which a planner on the vehicle side takes as input to plan a modified trajectory. In collaborative planning, the and negotiate a trajectory. Figures are taken from Brecht2024EvaluationOfConcepts
• Figure 4: The simulation environment used to evaluate the presented concept. It shows the ego vehicle during a lane change. Grey lines represent the lane topology. The lateral constraints are visualized as a red ($E_c^\text{left}$) and a green ($E_c^\text{right}$) line. The blue line represents the longitudinal constraint $\theta_\text{stop}$. The resulting trajectory is shown in dark green.
• Figure 5: Overview of interaction process between remote operator and automated driving system. The process starts with the IC becoming True if a disengagement is present. The remote operator is requested and supports the automation by providing a set of modified boundary conditions to the planner that plans a modified trajectory the remote operator approves. After trajectory execution, the remote operator checks if the situation was resolved, ending the teleoperation process by manually setting the CC to False.