Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Consensus-based optimization (CBO): Towards Global Optimality in Robotics

Xudong Sun, Armand Jordana, Massimo Fornasier, Jalal Etesami, Majid Khadiv

TL;DR

This paper introduces consensus-based optimization (CBO) to robotics, which is guaranteed to converge to a global optimum under mild assumptions, and opens a new framework to study global trajectory optimization in robotics.

Abstract

Zero-order optimization has recently received significant attention for designing optimal trajectories and policies for robotic systems. However, most existing methods (e.g., MPPI, CEM, and CMA-ES) are local in nature, as they rely on gradient estimation. In this paper, we introduce consensus-based optimization (CBO) to robotics, which is guaranteed to converge to a global optimum under mild assumptions. We provide theoretical analysis and illustrative examples that give intuition into the fundamental differences between CBO and existing methods. To demonstrate the scalability of CBO for robotics problems, we consider three challenging trajectory optimization scenarios: (1) a long-horizon problem for a simple system, (2) a dynamic balance problem for a highly underactuated system, and (3) a high-dimensional problem with only a terminal cost. Our results show that CBO is able to achieve lower costs with respect to existing methods on all three challenging settings. This opens a new framework to study global trajectory optimization in robotics.

Consensus-based optimization (CBO): Towards Global Optimality in Robotics

TL;DR

This paper introduces consensus-based optimization (CBO) to robotics, which is guaranteed to converge to a global optimum under mild assumptions, and opens a new framework to study global trajectory optimization in robotics.

Abstract

Zero-order optimization has recently received significant attention for designing optimal trajectories and policies for robotic systems. However, most existing methods (e.g., MPPI, CEM, and CMA-ES) are local in nature, as they rely on gradient estimation. In this paper, we introduce consensus-based optimization (CBO) to robotics, which is guaranteed to converge to a global optimum under mild assumptions. We provide theoretical analysis and illustrative examples that give intuition into the fundamental differences between CBO and existing methods. To demonstrate the scalability of CBO for robotics problems, we consider three challenging trajectory optimization scenarios: (1) a long-horizon problem for a simple system, (2) a dynamic balance problem for a highly underactuated system, and (3) a high-dimensional problem with only a terminal cost. Our results show that CBO is able to achieve lower costs with respect to existing methods on all three challenging settings. This opens a new framework to study global trajectory optimization in robotics.
Paper Structure (24 sections, 5 theorems, 55 equations, 8 figures, 1 algorithm)

This paper contains 24 sections, 5 theorems, 55 equations, 8 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Notations
  4. The big picture of global optimization
  5. zero-order methods used in robotics
  6. path integral methods
  7. Covariance matrix adaptation
  8. Consensus based optimization (CBO)
  9. Basics of CBO
  10. Advantages of CBO in global optimization
  11. Properties of CBO
  12. Experiment
  13. Experiment setting
  14. Long horizon planning
  15. Double Cartpole
  16. ...and 9 more sections

Key Result

Proposition 1

The update for the mean of distribution $\bar{u}_{0:T-1}$ under the PI framework in eq:pi_update can be written as where $\Delta(\bar{u}_{0:T-1})$ is the solution to the following constrained optimization problem with $\gamma$ being the Lagrange multiplier. where $\beta$ defines the Kullback–Leibler (KL) divergence metric between the current distribution and new distribution.

Figures (8)

  • Figure 1: An irregular, non-parametric distribution can focus probability on important directions by having longer tails, whereas a shrinking Gaussian is constrained by its fixed symmetric shape and has less flexibility.
  • Figure 2: A cost function landscape in a contour plot constructed with a unique global optimizer (marked star) and several local optimizers for illustration of how CBO works and its advantages. See text for more explanation.
  • Figure 3: Long horizon planning: comparison of the best loss across the population at each iteration for the long horizon planning problem. CBO wins by a large margin. CMA implemented according to \ref{['eq:cma_mean_ada']} and \ref{['eq:cma_cov_ada']}.
  • Figure 4: MPPI and CMA get stuck at a local minimum and fail to reach the tunnel. CBO succeeds in circumventing the left wall barriers and all obstacles (point mass agent) and reaches the tunnel. Each image shows the trajectory of a single experiment for different methods. Each column corresponds to one environment setting (spatial distribution of obstacles).
  • Figure 5: Double cartpole with cart mass $m_c= 1.0$ kg. Pole masses: $m_1 = m_2 = 0.1$ kg. Pole lengths: $l_1 = l_2 = 1.0$m.
  • ...and 3 more figures

Theorems & Definitions (24)

  • Remark 1
  • Remark 2
  • Remark 3
  • Proposition 1
  • Remark 4
  • Remark 5: Particle dynamic under parameterized distribution updates
  • Remark 6
  • Remark 7: Curse of dimensionality in finite sample approximation of Gaussian expectation
  • Remark 8: Connection between CMA and CEM
  • Remark 9: curse of distribution parametrization
  • ...and 14 more