Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Path planning with moving obstacles using stochastic optimal control

Seyyed Reza Jafari, Anders Hansson, Bo Wahlberg

TL;DR

This work tackles autonomous path planning in the presence of moving obstacles by casting the problem as an infinite-horizon stochastic optimal control problem (SSP) and addressing the curse of dimensionality via geometric symmetry. The authors derive a reduced three-dimensional state representation (e, d, theta) through SO(2) symmetry and solve a reduced Bellman equation with fitted value iteration, followed by online rollout to implement a receding-horizon controller. They introduce a two-term cost that balances fast target-reaching and safe separation from the obstacle, and demonstrate that the method outperforms control barrier function and receding-horizon A* approaches on average, with a clear trade-off controlled by lambda. The framework is validated numerically with a large discretization and sample-based value function approximation, showing substantial improvements in planning efficiency and safety in dynamic environments. Potential extensions include three-dimensional environments, moving targets, and multi-obstacle scenarios, as well as integration with learning-based methods in unknown settings.

Abstract

Navigating a collision-free, optimal path for a robot poses a perpetual challenge, particularly in the presence of moving objects such as humans. In this study, we formulate the problem of finding an optimal path as a stochastic optimal control problem. However, obtaining a solution to this problem is nontrivial. Therefore, we consider a simplified problem, which is more tractable. For this simplified formulation, we are able to solve the corresponding Bellman equation. However, the solution obtained from the simplified problem does not sufficiently address the original problem of interest. To address the full problem, we propose a numerical procedure where we solve an optimization problem at each sampling instant. The solution to the simplified problem is integrated into the online formulation as a final-state penalty. We illustrate the efficiency of the proposed method using a numerical example.

Path planning with moving obstacles using stochastic optimal control

TL;DR

This work tackles autonomous path planning in the presence of moving obstacles by casting the problem as an infinite-horizon stochastic optimal control problem (SSP) and addressing the curse of dimensionality via geometric symmetry. The authors derive a reduced three-dimensional state representation (e, d, theta) through SO(2) symmetry and solve a reduced Bellman equation with fitted value iteration, followed by online rollout to implement a receding-horizon controller. They introduce a two-term cost that balances fast target-reaching and safe separation from the obstacle, and demonstrate that the method outperforms control barrier function and receding-horizon A* approaches on average, with a clear trade-off controlled by lambda. The framework is validated numerically with a large discretization and sample-based value function approximation, showing substantial improvements in planning efficiency and safety in dynamic environments. Potential extensions include three-dimensional environments, moving targets, and multi-obstacle scenarios, as well as integration with learning-based methods in unknown settings.

Abstract

Navigating a collision-free, optimal path for a robot poses a perpetual challenge, particularly in the presence of moving objects such as humans. In this study, we formulate the problem of finding an optimal path as a stochastic optimal control problem. However, obtaining a solution to this problem is nontrivial. Therefore, we consider a simplified problem, which is more tractable. For this simplified formulation, we are able to solve the corresponding Bellman equation. However, the solution obtained from the simplified problem does not sufficiently address the original problem of interest. To address the full problem, we propose a numerical procedure where we solve an optimization problem at each sampling instant. The solution to the simplified problem is integrated into the online formulation as a final-state penalty. We illustrate the efficiency of the proposed method using a numerical example.

Paper Structure

This paper contains 13 sections, 1 theorem, 52 equations, 6 figures, 1 table, 1 algorithm.

Table of Contents

  1. INTRODUCTION
  2. Mathematical Model
  3. System Dynamics
  4. Stochastic Infinite Horizon Optimal Control Problem
  5. Solution of Optimal Control
  6. Geometric Symmetry
  7. fitted Value Iteration
  8. Effect of Probability Distribution and Constraints
  9. Numerical Results
  10. Conclusions
  11. Symmetry of Optimal Value Function in Stochastic Shortest Path Problems
  12. Dimension Reduction
  13. Dynamics of Reduced-States

Key Result

Theorem 1

Under the above assumptions, the optimal value function $V^\star$ is invariant under the group action and satisfies eq_A1.

Figures (6)

  • Figure 1: Geometrical representation of position of robot, and moving obstacle for two symmetric pairs $(h_1,r_1)$, and $(h_2,r_2)$
  • Figure 2: Geometrical representation of position of robot, obstacle, and target with future values
  • Figure 3: Trade-off curve between expected time to target (horizontal axis) and negative minimum distance to the obstacle (vertical axis) for different horizons, compared with receding horizon $A^*$, CBF, and CBF (CE) over the grid
  • Figure 4: Trade-off curve between expected time to target (horizontal axis) and negative minimum distance to the obstacle (vertical axis) for different horizons, for different horizons compared with receding-horizon $A^*$, CBF, and CBF (CE) for the given initial conditions
  • Figure 5: Trajectory of Robot, and Moving Obstacle using Different Methods
  • ...and 1 more figures

Theorems & Definitions (4)

  • Definition 1
  • Definition 2
  • Theorem 1
  • proof