Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Discrete-Time Maximum Likelihood Neural Distribution Steering

George Rapakoulias, Panagiotis Tsiotras

TL;DR

This paper studies the problem of steering the distribution of a discrete-time dynamical system from an initial distribution to a target distribution in finite time, and proposes an algorithm that results in a regularized maximum likelihood optimization problem, which is solved using machine learning techniques.

Abstract

This paper studies the problem of steering the distribution of a discrete-time dynamical system from an initial distribution to a target distribution in finite time. The formulation is fully nonlinear, allowing the use of general control policies, parametrized by neural networks. Although similar solutions have been explored in the continuous-time context, extending these techniques to systems with discrete dynamics is not trivial. The proposed algorithm results in a regularized maximum likelihood optimization problem, which is solved using machine learning techniques. After presenting the algorithm, we provide several numerical examples that illustrate the capabilities of the proposed method. We start from a simple problem that admits a solution through semidefinite programming, serving as a benchmark for the proposed approach. Then, we employ the framework in more general problems that cannot be solved using existing techniques, such as problems with non-Gaussian boundary distributions and non-linear dynamics.

Discrete-Time Maximum Likelihood Neural Distribution Steering

TL;DR

This paper studies the problem of steering the distribution of a discrete-time dynamical system from an initial distribution to a target distribution in finite time, and proposes an algorithm that results in a regularized maximum likelihood optimization problem, which is solved using machine learning techniques.

Abstract

This paper studies the problem of steering the distribution of a discrete-time dynamical system from an initial distribution to a target distribution in finite time. The formulation is fully nonlinear, allowing the use of general control policies, parametrized by neural networks. Although similar solutions have been explored in the continuous-time context, extending these techniques to systems with discrete dynamics is not trivial. The proposed algorithm results in a regularized maximum likelihood optimization problem, which is solved using machine learning techniques. After presenting the algorithm, we provide several numerical examples that illustrate the capabilities of the proposed method. We start from a simple problem that admits a solution through semidefinite programming, serving as a benchmark for the proposed approach. Then, we employ the framework in more general problems that cannot be solved using existing techniques, such as problems with non-Gaussian boundary distributions and non-linear dynamics.
Paper Structure (9 sections, 4 theorems, 20 equations, 4 figures, 1 table)

This paper contains 9 sections, 4 theorems, 20 equations, 4 figures, 1 table.

Table of Contents

  1. Intoduction
  2. Notation
  3. Preliminaries
  4. Normalizing flows
  5. Exact Steering Using Semidefinite Programming
  6. KL-divergence Covariance Steering
  7. Maximum likelihood Distribution Steering
  8. Numerical examples
  9. Conclusions

Key Result

Lemma 1

peyre2017computational (Change of Variables) Let x be an $n$-dimensional random variable with known PDF, denoted $p_x(x)$, and let $F: \mathbb{R}^n \rightarrow \mathbb{R}^n$ be a diffeomorphism. The PDF of $y = F(x)$, denoted $p_y(y)$, can be calculated using the formula

Figures (4)

  • Figure 1: (a) Exact SDP solution (b) Neural Network solution (c) Convergence plot along with optimal cost calculated using the SDP method. For figures (a), (b) the axes correspond to the first two states of the 2D double integrator.
  • Figure 2: GMM to Gaussian with given mean and covariance, double integrator prior dynamics and obstacles.
  • Figure 3: GT to Gaussian distribution steering with given mean and covariance with double integrator prior dynamics.
  • Figure 4: GMM to Gaussian distribution steering with nonlinear prior dynamics. The axes correspond to the system states.

Theorems & Definitions (6)

  • Lemma 1
  • Lemma 2
  • Lemma 3
  • Proposition 1
  • proof
  • Remark