Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Vision-based DRL Autonomous Driving Agent with Sim2Real Transfer

Dianzhao Li, Ostap Okhrin

TL;DR

This work proposes a vision-based deep reinforcement learning (DRL) agent that can simultaneously perform lane keeping and car following maneuvers and conducts a real-world evaluation to demonstrate the Sim2Real transfer capability of the trained DRL agent.

Abstract

To achieve fully autonomous driving, vehicles must be capable of continuously performing various driving tasks, including lane keeping and car following, both of which are fundamental and well-studied driving ones. However, previous studies have mainly focused on individual tasks, and car following tasks have typically relied on complete leader-follower information to attain optimal performance. To address this limitation, we propose a vision-based deep reinforcement learning (DRL) agent that can simultaneously perform lane keeping and car following maneuvers. To evaluate the performance of our DRL agent, we compare it with a baseline controller and use various performance metrics for quantitative analysis. Furthermore, we conduct a real-world evaluation to demonstrate the Sim2Real transfer capability of the trained DRL agent. To the best of our knowledge, our vision-based car following and lane keeping agent with Sim2Real transfer capability is the first of its kind.

Vision-based DRL Autonomous Driving Agent with Sim2Real Transfer

TL;DR

This work proposes a vision-based deep reinforcement learning (DRL) agent that can simultaneously perform lane keeping and car following maneuvers and conducts a real-world evaluation to demonstrate the Sim2Real transfer capability of the trained DRL agent.

Abstract

To achieve fully autonomous driving, vehicles must be capable of continuously performing various driving tasks, including lane keeping and car following, both of which are fundamental and well-studied driving ones. However, previous studies have mainly focused on individual tasks, and car following tasks have typically relied on complete leader-follower information to attain optimal performance. To address this limitation, we propose a vision-based deep reinforcement learning (DRL) agent that can simultaneously perform lane keeping and car following maneuvers. To evaluate the performance of our DRL agent, we compare it with a baseline controller and use various performance metrics for quantitative analysis. Furthermore, we conduct a real-world evaluation to demonstrate the Sim2Real transfer capability of the trained DRL agent. To the best of our knowledge, our vision-based car following and lane keeping agent with Sim2Real transfer capability is the first of its kind.
Paper Structure (18 sections, 8 equations, 8 figures, 4 tables, 1 algorithm)

This paper contains 18 sections, 8 equations, 8 figures, 4 tables, 1 algorithm.

Table of Contents

  1. INTRODUCTION
  2. Background
  3. Deep Reinforcement Learning
  4. Proximal Policy Optimization (PPO)
  5. Simulation and real-world platform
  6. Multi-task driving model with DRL
  7. Problem formulation
  8. Perception module for image processing
  9. Action and observation space
  10. Reward function
  11. Safety
  12. Driving efficiency
  13. Lateral and orientation deviation
  14. Experiments and results
  15. Training process
  16. ...and 3 more sections

Figures (8)

  • Figure 1: The proposed DRL framework for vision-based multi-task autonomous driving agents. The perception module leverages camera images to produce impact attributes regarding the environment, then the DRL control module utilizes the information to control the agent with enhanced generalization.
  • Figure 2: Robot car used during the real-world evaluation. (a) Side view of the robot car which equipped with front-view camera and Jetson Nano 2GB. (b) Back view of the car with a pattern of circles.
  • Figure 3: An example velocity trajectory for the leading vehicle generated by the Ornstein-Uhlenbeck process for the training process.
  • Figure 4: Training results of PPO agent with ten independent seeds over one million steps.
  • Figure 5: Example trajectories of DRL agent (red) and baseline controller (green) following random leader trajectory (orange) during the evaluation.
  • ...and 3 more figures