Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Twin Delayed Deep Deterministic Policy Gradient Algorithm for Autonomous Ground Vehicle Navigation via Digital Twin Perception Awareness

Kabirat Olayemi, Mien Van, Sean McLoone, Yuzhu Sun, Jack Close, Nguyen Minh Nhat, Stephen McIlvanna

TL;DR

A twin delayed deep deterministic policy gradient (TD3) algorithm that ensures collision avoidance and goal-based path planning and is capable of efficiently avoiding collision with obstacles and navigating to its desired destination, while at the same time safely avoids obstacles using the information received from the LIDAR sensor mounted on the robot.

Abstract

Autonomous ground vehicle (UGV) navigation has the potential to revolutionize the transportation system by increasing accessibility to disabled people, ensure safety and convenience of use. However, UGV requires extensive and efficient testing and evaluation to ensure its acceptance for public use. This testing are mostly done in a simulator which result to sim2real transfer gap. In this paper, we propose a digital twin perception awareness approach for the control of robot navigation without prior creation of the virtual environment (VT) environment state. To achieve this, we develop a twin delayed deep deterministic policy gradient (TD3) algorithm that ensures collision avoidance and goal-based path planning. We demonstrate the performance of our approach on different environment dynamics. We show that our approach is capable of efficiently avoiding collision with obstacles and navigating to its desired destination, while at the same time safely avoids obstacles using the information received from the LIDAR sensor mounted on the robot. Our approach bridges the gap between sim-to-real transfer and contributes to the adoption of UGVs in real world. We validate our approach in simulation and a real-world application in an office space.

A Twin Delayed Deep Deterministic Policy Gradient Algorithm for Autonomous Ground Vehicle Navigation via Digital Twin Perception Awareness

TL;DR

A twin delayed deep deterministic policy gradient (TD3) algorithm that ensures collision avoidance and goal-based path planning and is capable of efficiently avoiding collision with obstacles and navigating to its desired destination, while at the same time safely avoids obstacles using the information received from the LIDAR sensor mounted on the robot.

Abstract

Autonomous ground vehicle (UGV) navigation has the potential to revolutionize the transportation system by increasing accessibility to disabled people, ensure safety and convenience of use. However, UGV requires extensive and efficient testing and evaluation to ensure its acceptance for public use. This testing are mostly done in a simulator which result to sim2real transfer gap. In this paper, we propose a digital twin perception awareness approach for the control of robot navigation without prior creation of the virtual environment (VT) environment state. To achieve this, we develop a twin delayed deep deterministic policy gradient (TD3) algorithm that ensures collision avoidance and goal-based path planning. We demonstrate the performance of our approach on different environment dynamics. We show that our approach is capable of efficiently avoiding collision with obstacles and navigating to its desired destination, while at the same time safely avoids obstacles using the information received from the LIDAR sensor mounted on the robot. Our approach bridges the gap between sim-to-real transfer and contributes to the adoption of UGVs in real world. We validate our approach in simulation and a real-world application in an office space.
Paper Structure (18 sections, 6 equations, 7 figures, 2 tables, 3 algorithms)

This paper contains 18 sections, 6 equations, 7 figures, 2 tables, 3 algorithms.

Table of Contents

  1. INTRODUCTION
  2. Related Work
  3. Background
  4. Methodology
  5. Reinforcement Learning Pre-trained Algorithm
  6. State Space
  7. Action Space
  8. Transition Probability
  9. Reward Function
  10. Discount Factor
  11. Digital Twin
  12. Reinforcement Learning Re-training Algorithm
  13. Results
  14. Simulation Performance
  15. Performance without Digital Twin Retraining
  16. ...and 3 more sections

Figures (7)

  • Figure 1: Illustration of the VT gazebo environment creation based of the information received from the LIDAR sensor pointcloud data. The leftmost figure shows the environment and it's corresponding poincloud representation in RVIS is shown in the middle figure. The leftmost figure VT of the created from the processed LIDAR data. The robot shown is an unmanned ground vehicle (HUSKY A200) developed by ClearPath. In the gazebo, all obstacles at a distance greater than 1m from the robot are represented in green, while obstacles less than $2m$ from the robot are in red.
  • Figure 2: Digital twin key components for real-time data integration.
  • Figure 3: Digital twin key components for real-time data integration.
  • Figure 4: Digital twin key components for real-time data integration.
  • Figure 5: Sample of trajectories generated in the simulation environment. The initial position of the robot is 0.0 on both x and y axis for the initial test scenarios while for other test scenarios it is the goal of the previous test scenarios. Each green point represents the target the robot navigated to, while the broken lines represent the path to each goal.
  • ...and 2 more figures