Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Line-Search Filter Differential Dynamic Programming for Optimal Control with Nonlinear Equality Constraints

Ming Xu, Stephen Gould, Iman Shames

TL;DR

This work introduces FilterDDP, a line-search filter-based differential dynamic programming method for discrete-time optimal control problems with nonlinear equality constraints in robotics. It replaces traditional merit-function or augmented-Lagrangian strategies with a Lagrangian-based step acceptance criterion and a perturbation of the value-function Hessian, providing a formal proof of local quadratic convergence and a primal-dual interior-point extension for inequalities. The algorithm alternates between backward and forward passes, uses a perturbed Newton step to compute updates, and employs a backtracking line search with a dynamic filter to ensure robust convergence. Numerical experiments on contact-implicit robotics tasks demonstrate improved robustness, faster per-iteration performance, and reliable convergence compared with IPOPT and ProxDDP, highlighting the practical impact for constrained trajectory optimization in embedded robotic systems.

Abstract

We present FilterDDP, a differential dynamic programming algorithm for solving discrete-time, optimal control problems (OCPs) with nonlinear equality constraints. Unlike prior methods based on merit functions or the augmented Lagrangian class of algorithms, FilterDDP uses a step filter in conjunction with a line search to handle equality constraints. We identify two important design choices for the step filter criteria which lead to robust numerical performance: 1) we use the Lagrangian instead of the cost in the step acceptance criterion and, 2) in the backward pass, we perturb the value function Hessian. Both choices are rigorously justified, for 2) in particular by a formal proof of local quadratic convergence. In addition to providing a primal-dual interior point extension for handling OCPs with both equality and inequality constraints, we validate FilterDDP on three contact implicit trajectory optimisation problems which arise in robotics.

Line-Search Filter Differential Dynamic Programming for Optimal Control with Nonlinear Equality Constraints

TL;DR

This work introduces FilterDDP, a line-search filter-based differential dynamic programming method for discrete-time optimal control problems with nonlinear equality constraints in robotics. It replaces traditional merit-function or augmented-Lagrangian strategies with a Lagrangian-based step acceptance criterion and a perturbation of the value-function Hessian, providing a formal proof of local quadratic convergence and a primal-dual interior-point extension for inequalities. The algorithm alternates between backward and forward passes, uses a perturbed Newton step to compute updates, and employs a backtracking line search with a dynamic filter to ensure robust convergence. Numerical experiments on contact-implicit robotics tasks demonstrate improved robustness, faster per-iteration performance, and reliable convergence compared with IPOPT and ProxDDP, highlighting the practical impact for constrained trajectory optimization in embedded robotic systems.

Abstract

We present FilterDDP, a differential dynamic programming algorithm for solving discrete-time, optimal control problems (OCPs) with nonlinear equality constraints. Unlike prior methods based on merit functions or the augmented Lagrangian class of algorithms, FilterDDP uses a step filter in conjunction with a line search to handle equality constraints. We identify two important design choices for the step filter criteria which lead to robust numerical performance: 1) we use the Lagrangian instead of the cost in the step acceptance criterion and, 2) in the backward pass, we perturb the value function Hessian. Both choices are rigorously justified, for 2) in particular by a formal proof of local quadratic convergence. In addition to providing a primal-dual interior point extension for handling OCPs with both equality and inequality constraints, we validate FilterDDP on three contact implicit trajectory optimisation problems which arise in robotics.

Paper Structure

This paper contains 32 sections, 5 theorems, 42 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Notation used in this paper
  3. Discrete-time Optimal Control
  4. Problem Formulation
  5. Optimality Conditions
  6. Background and Derivation of DDP
  7. Dynamic Programming for Optimal Control
  8. Differential Dynamic Programming
  9. Filter Differential Dynamic Programming
  10. Termination Criterion
  11. Backward Pass
  12. Update Rule
  13. Updating the Value Function Approximation
  14. Differences to prior works e.g., proxddppavlov_ipddpaltro
  15. Forward Pass
  16. ...and 17 more sections

Key Result

Theorem 1

A value function $V^t$ which has associated first and second-order derivatives given by eq:vxupdate and $\bar{V}_{xx}^t = \bar{Q}_{xx}^t + \beta_t^\top \bar{Q}_{uu}^t \beta_t + \bar{Q}_{xu}^t \beta_t + \beta_t^\top \bar{Q}_{ux}^t$ (i.e., unperturbed eq:vxxupdate), respectively, is given by $V^{N+1}( Note that $V^1(\hat{x}_1, \gamma) = \mathcal{L}(\mathbf{w}^+(\gamma))$, i.e., the updated $\mathcal

Figures (4)

  • Figure 1: Results for the three planning tasks. For a)-c), the x-axis represents iteration count and the y-axis is the number of OCPs which converged to the error tolerance of $10^{-7}$ for Filter DDP and IPOPT and $10^{-5}$ for ProxDDP and IPOPT (B).
  • Figure 2: Acrobot example. $s_t$ is the signed distance to the $\pi/2$ and $-\pi/2$ joint limits and $\lambda_t$ denotes the impulses. Around the 4s mark, the acrobot is "leaning into" the joint limit.
  • Figure 3: Pusher and slider trajectories for two instances of the non-prehensile manipulation problem. The pusher and slider trajectories are marked by the blue and brown lines.
  • Figure 4: Local quadratic convergence of FilterDDP. The x-axis measures iteration count and the y-axis measures $\|\bar{u}_{1:N} - u^\star_{1:N}\|_2$, where $u^\star_{1:N}$ is the optimal point found by FilterDDP.

Theorems & Definitions (13)

  • Theorem 1
  • proof
  • Corollary 1
  • proof
  • Theorem 2
  • proof
  • Proposition 1
  • proof
  • Remark 1
  • Theorem 3
  • ...and 3 more