Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Collision probability reduction method for tracking control in automatic docking / berthing using reinforcement learning

Kouki Wakita, Youhei Akimoto, Dimas M. Rachman, Yoshiki Miyauchi, Umeda Naoya, Atsuo Maki

TL;DR

A training method based on reinforcement learning for a trajectory tracking controller that reduces the probability of collisions with static obstacles and shows the tracking performance in a model experiment.

Abstract

Automation of berthing maneuvers in shipping is a pressing issue as the berthing maneuver is one of the most stressful tasks seafarers undertake. Berthing control problems are often tackled via tracking a predefined trajectory or path. Maintaining a tracking error of zero under an uncertain environment is impossible; the tracking controller is nonetheless required to bring vessels close to desired berths. The tracking controller must prioritize the avoidance of tracking errors that may cause collisions with obstacles. This paper proposes a training method based on reinforcement learning for a trajectory tracking controller that reduces the probability of collisions with static obstacles. Via numerical simulations, we show that the proposed method reduces the probability of collisions during berthing maneuvers. Furthermore, this paper shows the tracking performance in a model experiment.

Collision probability reduction method for tracking control in automatic docking / berthing using reinforcement learning

TL;DR

A training method based on reinforcement learning for a trajectory tracking controller that reduces the probability of collisions with static obstacles and shows the tracking performance in a model experiment.

Abstract

Automation of berthing maneuvers in shipping is a pressing issue as the berthing maneuver is one of the most stressful tasks seafarers undertake. Berthing control problems are often tackled via tracking a predefined trajectory or path. Maintaining a tracking error of zero under an uncertain environment is impossible; the tracking controller is nonetheless required to bring vessels close to desired berths. The tracking controller must prioritize the avoidance of tracking errors that may cause collisions with obstacles. This paper proposes a training method based on reinforcement learning for a trajectory tracking controller that reduces the probability of collisions with static obstacles. Via numerical simulations, we show that the proposed method reduces the probability of collisions during berthing maneuvers. Furthermore, this paper shows the tracking performance in a model experiment.
Paper Structure (20 sections, 23 equations, 17 figures, 10 tables)

This paper contains 20 sections, 23 equations, 17 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Research
  3. Scope of this study
  4. Method
  5. Optimization of the trajectory tracking controller
  6. Training Environment
  7. Desired trajectory and pseudo-obstacle
  8. Maneuvering simulation
  9. Input of the controller
  10. Reward function
  11. Application to berthing maneuvers
  12. Trajectory Planning
  13. Selection of the desired state
  14. Pseudo-obstacles in the target harbor
  15. Results
  16. ...and 5 more sections

Figures (17)

  • Figure 1: Subject ship in Inukai pond
  • Figure 2: The coordinates systems used in this work.
  • Figure 3: The generation method of static pseudo-obstacles used in this work.
  • Figure 4: An example of a desired trajectory with the static pseudo-obstacles used in training.
  • Figure 5: Illustration of the relationship between the vessel and static obstacle
  • ...and 12 more figures