Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Three-Dimensional Pursuit-Evasion Game Based on Fuzzy Actor-Critic Learning Algorithm

Penglin Hu

TL;DR

This work tackles pursuit-evasion in three-dimensional space by extending the Apollonius construct to a generalized 3D framework and deriving an optimal motion cone for pursuers and evaders. It presents a fuzzy actor-critic learning (FACL) approach with Takagi-Sugeno rules and a reward function based on artificial potential fields to train agents in continuous 3D action spaces. Validation in simulations shows successful captures, shorter end-to-end distances, and improved path length and capture time relative to a prior 3D PEG method, demonstrating enhanced pursuit performance and obstacle handling. Overall, the paper lays groundwork for more scalable, multi-agent and multi-objective RL strategies in 3D PEG settings.

Abstract

Most of the existing research on pursuit-evasion game (PEG) is conducted in a two-dimensional (2D) environment. In this paper, we investigate the PEG in a 3D space. We extend the Apollonius circle (AC) to the 3D space and introduce its detailed analytical form. To enhance the capture efficiency, we derive the optimal motion space for both the pursuer and the evader. To address the issue arising from a discrete state space, we design a fuzzy actor-critic learning (FACL) algorithm to obtain the agents' strategies. To improve learning performance, we devise a reward function for the agents, which enables obstacle avoidance functionality. The effectiveness of the proposed algorithm is validated through simulation experiments.

A Three-Dimensional Pursuit-Evasion Game Based on Fuzzy Actor-Critic Learning Algorithm

TL;DR

This work tackles pursuit-evasion in three-dimensional space by extending the Apollonius construct to a generalized 3D framework and deriving an optimal motion cone for pursuers and evaders. It presents a fuzzy actor-critic learning (FACL) approach with Takagi-Sugeno rules and a reward function based on artificial potential fields to train agents in continuous 3D action spaces. Validation in simulations shows successful captures, shorter end-to-end distances, and improved path length and capture time relative to a prior 3D PEG method, demonstrating enhanced pursuit performance and obstacle handling. Overall, the paper lays groundwork for more scalable, multi-agent and multi-objective RL strategies in 3D PEG settings.

Abstract

Most of the existing research on pursuit-evasion game (PEG) is conducted in a two-dimensional (2D) environment. In this paper, we investigate the PEG in a 3D space. We extend the Apollonius circle (AC) to the 3D space and introduce its detailed analytical form. To enhance the capture efficiency, we derive the optimal motion space for both the pursuer and the evader. To address the issue arising from a discrete state space, we design a fuzzy actor-critic learning (FACL) algorithm to obtain the agents' strategies. To improve learning performance, we devise a reward function for the agents, which enables obstacle avoidance functionality. The effectiveness of the proposed algorithm is validated through simulation experiments.

Paper Structure

This paper contains 8 sections, 20 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Proposed Method
  4. Generalized AC in 3D Space
  5. The Fuzzy Actor-Critic Learning Algorithm
  6. Reward Function Based on Artificial Potential Field
  7. Simulation and Result
  8. Conclusions

Figures (5)

  • Figure 1: Illustration of the PEG model, where $P_t$ represents the pursuer, $E_t$ represents the evader, and the cylinder represents the obstacle $O$
  • Figure 2: Illustration of the generalized AC, where $P=(10,10,10)$, $E=(-10,-10,10)$, and $O_{AC}$ represent the pursuer, evader, and the center of the generalized AC respectively. The yellow sphere indicates that $a<1$, the blue surface indicates that $a=1$, and the green surface indicates that $a > 1$
  • Figure 3: Illustration of the generalized AC with $a<1$
  • Figure 4: Trajectories of agents in the 3D environment
  • Figure 5: The path length and capture time of two algorithms