Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Autonomous Vehicle Decision-Making Framework for Considering Malicious Behavior at Unsignalized Intersections

Qing Li, Jinxing Hua, Qiuxia Sun

TL;DR

Safety gains are modulated by variable weighting parameters to ensure that safety can be emphasized more in emergency situations and enables Autonomous Vehicles to make informed decisions when encountering vehicles with potentially malicious behaviors at unsignalized intersections.

Abstract

In this paper, we propose a Q-learning based decision-making framework to improve the safety and efficiency of Autonomous Vehicles when they encounter other maliciously behaving vehicles while passing through unsignalized intersections. In Autonomous Vehicles, conventional reward signals are set as regular rewards regarding feedback factors such as safety and efficiency. In this paper, safety gains are modulated by variable weighting parameters to ensure that safety can be emphasized more in emergency situations. The framework proposed in this paper introduces first-order theory of mind inferences on top of conventional rewards, using first-order beliefs as additional reward signals. The decision framework enables Autonomous Vehicles to make informed decisions when encountering vehicles with potentially malicious behaviors at unsignalized intersections, thereby improving the overall safety and efficiency of Autonomous Vehicle transportation systems. In order to verify the performance of the decision framework, this paper uses Prescan/Simulink co-simulations for simulation, and the results show that the performance of the decision framework can meet the set requirements.

Autonomous Vehicle Decision-Making Framework for Considering Malicious Behavior at Unsignalized Intersections

TL;DR

Safety gains are modulated by variable weighting parameters to ensure that safety can be emphasized more in emergency situations and enables Autonomous Vehicles to make informed decisions when encountering vehicles with potentially malicious behaviors at unsignalized intersections.

Abstract

In this paper, we propose a Q-learning based decision-making framework to improve the safety and efficiency of Autonomous Vehicles when they encounter other maliciously behaving vehicles while passing through unsignalized intersections. In Autonomous Vehicles, conventional reward signals are set as regular rewards regarding feedback factors such as safety and efficiency. In this paper, safety gains are modulated by variable weighting parameters to ensure that safety can be emphasized more in emergency situations. The framework proposed in this paper introduces first-order theory of mind inferences on top of conventional rewards, using first-order beliefs as additional reward signals. The decision framework enables Autonomous Vehicles to make informed decisions when encountering vehicles with potentially malicious behaviors at unsignalized intersections, thereby improving the overall safety and efficiency of Autonomous Vehicle transportation systems. In order to verify the performance of the decision framework, this paper uses Prescan/Simulink co-simulations for simulation, and the results show that the performance of the decision framework can meet the set requirements.
Paper Structure (21 sections, 21 equations, 15 figures, 1 table, 1 algorithm)

This paper contains 21 sections, 21 equations, 15 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Decision-making design
  3. Definition of vehicles
  4. State space
  5. Action space
  6. Reward
  7. Safety payoff function
  8. Efficiency payoff function
  9. Comfort payoff function
  10. Total compensation function
  11. Q-value function
  12. Learning and strategy selection
  13. Malicious behavior simulation
  14. Simulation results and discussion
  15. Using only the game model
  16. ...and 6 more sections

Figures (15)

  • Figure 1: Decision-Making Framework for Autonomous Vehicles at Unsignalized Intersections
  • Figure 2: Schematic diagram of intersection
  • Figure 3: (a) No theory of mind extrapolation (b) First-order theory of mind extrapolation (c) Second-order theory of mind
  • Figure 4: The structure of the simulated environment
  • Figure 5: Schematic of vehicle behavior at unsignalized intersections
  • ...and 10 more figures