Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Maximum Entropy Density Control of Discrete-Time Linear Systems with Quadratic Cost

Kaito Ito, Kenji Kashima

TL;DR

This work develops a complete solution to maximum entropy density control for discrete-time linear systems with Gaussian endpoints and quadratic costs. By formulating the problem as an entropy-regularized LQ control, the authors derive a pair of coupled Riccati difference equations that uniquely determine the optimal policy, which is a time-varying Gaussian with a linear state-feedback mean plus an independent noise term. They further show an equivalent backward (time-reversed) density-control problem and establish a Schrödinger-bridge–like duality, enabling simultaneous forward and backward solutions. In the zero-entropy limit, the results converge to a closed-form unregularized density-control policy, clarifying the connection to standard covariance and mean steering. A numerical example illustrates how the theory shapes trajectories and covariances under different cost weights and regularization levels, underscoring practical applicability and robustness considerations.

Abstract

This paper addresses the problem of steering the distribution of the state of a discrete-time linear system to a given target distribution while minimizing an entropy-regularized cost functional. This problem is called a maximum entropy density control problem. Specifically, the running cost is given by quadratic forms of the state and the control input, and the initial and target distributions are Gaussian. We first reveal that our problem boils down to solving two Riccati difference equations coupled through their boundary values. Based on them, we give the closed-form expression of the unique optimal policy. Next, we show that the optimal density control of a backward system can be obtained simultaneously with the forward-time optimal policy. The backward solution gives another expression of the forward solution. Finally, by considering the limit where the entropy regularization vanishes, we derive the unregularized density control in closed form.

Maximum Entropy Density Control of Discrete-Time Linear Systems with Quadratic Cost

TL;DR

This work develops a complete solution to maximum entropy density control for discrete-time linear systems with Gaussian endpoints and quadratic costs. By formulating the problem as an entropy-regularized LQ control, the authors derive a pair of coupled Riccati difference equations that uniquely determine the optimal policy, which is a time-varying Gaussian with a linear state-feedback mean plus an independent noise term. They further show an equivalent backward (time-reversed) density-control problem and establish a Schrödinger-bridge–like duality, enabling simultaneous forward and backward solutions. In the zero-entropy limit, the results converge to a closed-form unregularized density-control policy, clarifying the connection to standard covariance and mean steering. A numerical example illustrates how the theory shapes trajectories and covariances under different cost weights and regularization levels, underscoring practical applicability and robustness considerations.

Abstract

This paper addresses the problem of steering the distribution of the state of a discrete-time linear system to a given target distribution while minimizing an entropy-regularized cost functional. This problem is called a maximum entropy density control problem. Specifically, the running cost is given by quadratic forms of the state and the control input, and the initial and target distributions are Gaussian. We first reveal that our problem boils down to solving two Riccati difference equations coupled through their boundary values. Based on them, we give the closed-form expression of the unique optimal policy. Next, we show that the optimal density control of a backward system can be obtained simultaneously with the forward-time optimal policy. The backward solution gives another expression of the forward solution. Finally, by considering the limit where the entropy regularization vanishes, we derive the unregularized density control in closed form.
Paper Structure (13 sections, 16 theorems, 134 equations, 11 figures)

This paper contains 13 sections, 16 theorems, 134 equations, 11 figures.

Table of Contents

  1. Introduction
  2. Problem Formulation and Preliminary Analysis
  3. Solution to MaxEnt Density Control Problem
  4. Equivalent Backward Density Control Problem
  5. Unregularized Density Control as the Zero-Noise Limit of MaxEnt Density Control
  6. Example
  7. Conclusion
  8. Proof of Proposition \ref{['prop:riccati']}
  9. Proof of Proposition \ref{['prop:linear']}
  10. Lemmas for the Proofs of Propositions \ref{['thm:riccati_solution']}, \ref{['prop:positive']}
  11. Proof of Proposition \ref{['thm:riccati_solution']}
  12. Proof of Proposition \ref{['prop:positive']}
  13. Proof of Proposition \ref{['prop:opt_inv']} and \ref{['eq:opt_forward_another']}

Key Result

Proposition 1

Assume that $\Pi_k \in {\mathcal{S}}^n$ satisfies $R_k + B_k^\top \Pi_{k+1} B_k \succ 0$ for any $k\in [\![0,N-1]\!]$ and is a solution to the following Riccati difference equation: Then, the unique optimal policy of Problem prob:LQ_control is given by for any $k \in [\![0,N-1]\!] , \ u\in {\mathbb R}^m,$ and $h_k \in {\mathbb H}_k$. In addition, the minimum value of eq:cost_lq is given by

Figures (11)

  • Figure 1: $Q_k = I$
  • Figure 2: $Q_k = 10I$
  • Figure 3: $Q_k = I$
  • Figure 4: $Q_k = 10I$
  • Figure 5: $Q_k = I$
  • ...and 6 more figures

Theorems & Definitions (26)

  • Proposition 1
  • proof
  • Proposition 2
  • Proposition 3
  • Remark 1
  • Proposition 4
  • Proposition 5
  • Proposition 6
  • Theorem 1
  • Remark 2
  • ...and 16 more