Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Decision-Making GPT Model Augmented with Entropy Regularization for Autonomous Vehicles

Jiaqi Liu, Shiyu Fang, Xuekai Liu, Lulu Guo, Peng Hang, Jian Sun

TL;DR

A novel decision-making model tailored for AVs is introduced, which incorporates entropy regularization techniques to bolster exploration and enhance safety performance and surpasses several established baseline methods in terms of safety and overall efficacy.

Abstract

In the domain of autonomous vehicles (AVs), decision-making is a critical factor that significantly influences the efficacy of autonomous navigation. As the field progresses, the enhancement of decision-making capabilities in complex environments has become a central area of research within data-driven methodologies. Despite notable advances, existing learning-based decision-making strategies in autonomous vehicles continue to reveal opportunities for further refinement, particularly in the articulation of policies and the assurance of safety. In this study, the decision-making challenges associated with autonomous vehicles are conceptualized through the framework of the Constrained Markov Decision Process (CMDP) and approached as a sequence modeling problem. Utilizing the Generative Pre-trained Transformer (GPT), we introduce a novel decision-making model tailored for AVs, which incorporates entropy regularization techniques to bolster exploration and enhance safety performance. Comprehensive experiments conducted across various scenarios affirm that our approach surpasses several established baseline methods, particularly in terms of safety and overall efficacy.

A Decision-Making GPT Model Augmented with Entropy Regularization for Autonomous Vehicles

TL;DR

A novel decision-making model tailored for AVs is introduced, which incorporates entropy regularization techniques to bolster exploration and enhance safety performance and surpasses several established baseline methods in terms of safety and overall efficacy.

Abstract

In the domain of autonomous vehicles (AVs), decision-making is a critical factor that significantly influences the efficacy of autonomous navigation. As the field progresses, the enhancement of decision-making capabilities in complex environments has become a central area of research within data-driven methodologies. Despite notable advances, existing learning-based decision-making strategies in autonomous vehicles continue to reveal opportunities for further refinement, particularly in the articulation of policies and the assurance of safety. In this study, the decision-making challenges associated with autonomous vehicles are conceptualized through the framework of the Constrained Markov Decision Process (CMDP) and approached as a sequence modeling problem. Utilizing the Generative Pre-trained Transformer (GPT), we introduce a novel decision-making model tailored for AVs, which incorporates entropy regularization techniques to bolster exploration and enhance safety performance. Comprehensive experiments conducted across various scenarios affirm that our approach surpasses several established baseline methods, particularly in terms of safety and overall efficacy.
Paper Structure (16 sections, 16 equations, 4 figures, 2 tables, 1 algorithm)

This paper contains 16 sections, 16 equations, 4 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Constrained Markov Decision Process (CMDP) and Safe RL
  4. Problem Formulation
  5. State Space
  6. Action Space
  7. Methodology
  8. Decision-making GPT Model
  9. Input Representation with Target Cost Threshold
  10. Architecture
  11. Sequence Modeling with Entropy Regularization
  12. Experiment and Evaluation
  13. Experiments and Baselines
  14. Implementation Details
  15. Results Analysis
  16. ...and 1 more sections

Figures (4)

  • Figure 1: The overall procedure of our work.
  • Figure 2: The workflow of our decision-making GPT model.
  • Figure 3: Three kinds of scenarios we used for our methods training and testing.
  • Figure 4: The reward and cost results of our method and baselines from different scenarios. Subfigures (a) and (d) are from scenario 1; subfigures (b) and (e) are from scenario 2; subfigures (c) and (f) are from scenario 3.