Speed Optimization Algorithm based on Deterministic Markov Decision Process for Automated Highway Merge

TL;DR

The paper tackles safe and efficient ramp merging for automated highway driving. It formulates speed planning as a deterministic MDP with state $S=T×G×V×L$ and action set $A={j_{-n_j},…,j_0…,j_{n_j}}$, using a deterministic transition $f(s,a)$ and jerk-based controls. Key equations include the action-value function $Q(s,a)=r(s,a,f(s,a))+γ V(f(s,a))$, the policy $π(s)=argmax_a Q(s,a)$, and the value function $V(s)=max_a V(f(s,a))$, with termination set $S_T=S_{T_e}∪S_{T_p}$. Results demonstrate real-time performance with a planning step of $0.1s$ and improved merging behavior over an Intelligent Driver Model baseline in both simulation and real-vehicle tests. The work provides a viable real-time deterministic MDP framework for highway merges and lays a foundation for extensions to urban scenarios and reward-function refinement.

Abstract

This study presents a robust optimization algorithm for automated highway merge. The merging scenario is one of the challenging scenes in automated driving, because it requires adjusting ego vehicle's speed to match other vehicles before reaching the end point. Then, we model the speed planning problem as a deterministic Markov decision process. The proposed scheme is able to compute each state value of the process and reliably derive the optimal sequence of actions. In our approach, we adopt jerk as the action of the process to prevent a sudden change of acceleration. However, since this expands the state space, we also consider ways to achieve a real-time operation. We compared our scheme with a simple algorithm with the Intelligent Driver Model. We not only evaluated the scheme in a simulation environment but also conduct a real world testing.

Figures (8)

• Figure 1: Example of the state transition.
• Figure 2: The state which is terminated by the limit speed.
• Figure 3: The state which is terminated and the state whose reward is attenuated by other vehicles.
• Figure 4: The course where the merging test is conducted.
• Figure 5: The result of position of trajectory.