Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Differential dynamic programming with stagewise equality and inequality constraints using interior point method

Siddharth Prabhu, Srinivas Rangarajan, Mayuresh Kothare

TL;DR

This work extends Differential Dynamic Programming to handle arbitrary stagewise equality and inequality constraints by embedding an interior point barrier framework. It derives explicit Newton-based update rules for the control, Lagrange multipliers, and slack variables, and uses regularization to ensure a PSD reduced Hessian. The algorithm solves a sequence of barrier subproblems via backward and forward passes with line-search to guarantee convergence, updating the barrier parameter $\tau$ until optimality is approached. Experiments on an inverted pendulum, a continuous stirred tank reactor, car parking, and obstacle avoidance show successful constraint satisfaction and feasible trajectories under long horizons, validating the practical viability of constrained IP-DDP. The approach can be extended to alternative solvers like iLQR or to multiple shooting variants for broader applicability.

Abstract

Differential Dynamic Programming (DDP) is one of the indirect methods for solving an optimal control problem. Several extensions to DDP have been proposed to add stagewise state and control constraints, which can mainly be classified as augmented lagrangian methods, active set methods, and barrier methods. In this paper, we use an interior point method, which is a type of barrier method, to incorporate arbitrary stagewise equality and inequality state and control constraints. We also provide explicit update formulas for all the involved variables. Finally, we apply this algorithm to example systems such as the inverted pendulum, a continuously stirred tank reactor, car parking, and obstacle avoidance.

Differential dynamic programming with stagewise equality and inequality constraints using interior point method

TL;DR

This work extends Differential Dynamic Programming to handle arbitrary stagewise equality and inequality constraints by embedding an interior point barrier framework. It derives explicit Newton-based update rules for the control, Lagrange multipliers, and slack variables, and uses regularization to ensure a PSD reduced Hessian. The algorithm solves a sequence of barrier subproblems via backward and forward passes with line-search to guarantee convergence, updating the barrier parameter until optimality is approached. Experiments on an inverted pendulum, a continuous stirred tank reactor, car parking, and obstacle avoidance show successful constraint satisfaction and feasible trajectories under long horizons, validating the practical viability of constrained IP-DDP. The approach can be extended to alternative solvers like iLQR or to multiple shooting variants for broader applicability.

Abstract

Differential Dynamic Programming (DDP) is one of the indirect methods for solving an optimal control problem. Several extensions to DDP have been proposed to add stagewise state and control constraints, which can mainly be classified as augmented lagrangian methods, active set methods, and barrier methods. In this paper, we use an interior point method, which is a type of barrier method, to incorporate arbitrary stagewise equality and inequality state and control constraints. We also provide explicit update formulas for all the involved variables. Finally, we apply this algorithm to example systems such as the inverted pendulum, a continuously stirred tank reactor, car parking, and obstacle avoidance.
Paper Structure (14 sections, 38 equations, 4 figures, 5 algorithms)

This paper contains 14 sections, 38 equations, 4 figures, 5 algorithms.

Table of Contents

  1. Introduction
  2. Differential Dynamic Programming
  3. Preliminaries
  4. Constrained Differential Dynamic Programming
  5. Regularization
  6. Line Search
  7. Convergence criteria
  8. Algorithm
  9. Experiments
  10. Inverted Pendulum
  11. Continuous Stirred Tank Reactor
  12. Car Parking
  13. Car Obstacle
  14. Conclusion

Figures (4)

  • Figure 1: Solution of the inverted pendulum problem given by algorithm \ref{['algo:ipddp']}
  • Figure 2: Solution of the continuous stirred tank reactor problem given by algorithm \ref{['algo:ipddp']}
  • Figure 3: Solution of the car parking problem given by algorithm \ref{['algo:ipddp']}
  • Figure 4: Solution of the car obstacle problem given by algorithm \ref{['algo:ipddp']}