Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Information-Theoretic Analysis of Discrete-Time Control and Filtering Limitations by the I-MMSE Relationships

Neng Wan, Dapeng Li, Naira Hovakimyan, Petros G. Voulgaris

TL;DR

This work extends the I-MMSE relationship to discrete-time AWGN channels with and without feedback and uses the resulting total information rate as a unifying metric for control and filtering trade-offs. By modeling control and filtering tasks as Gaussian channels and leveraging optimal estimation via causal/predictive MMSE, it derives sandwich bounds and exact equalities that connect information rate to unstable dynamics and Bode-type integrals, for LTI, LTV, and nonlinear systems. For control, it shows $\bar{I}(E; X_0)=\sum_j \log |\lambda_j^+(A)|$ in the LTI case and $\mathscr{B}$ in the LTV case, with nonlinear cases bounded via MMSE-based estimates; for filtering, it proves $\bar{I}(Y; X_0)=\mathscr{B}$ (LTV) or $\sum_j \log|\lambda_j^+(A)|$ (LTI) under vanishing process noise, providing a lower bound on the best achievable MMSE. Collectively, the results offer an information-theoretic lens on time-averaged control and estimation limits and yield practical ways to bound and estimate fundamental trade-offs, with extensions to colored or non-Gaussian disturbances and to broader channel models as future work.

Abstract

Fundamental limitations or performance trade-offs/limits are important properties and constraints of both control and filtering systems. Among various trade-off metrics, total information rate that characterizes the sensitivity trade-offs and time-averaged performance of control and filtering systems was conventionally studied by using the differential entropy rate and Kolmogorov-Bode formula. In this paper, by extending the famous I-MMSE (mutual information -- minimum mean-square error) relationships to the discrete-time additive white Gaussian channels with and without feedback, a new paradigm is introduced to estimate and analyze total information rate as a control and filtering trade-off metric. Under this framework, we explore the trade-off properties of total information rate for a variety of the discrete-time control and filtering systems, e.g., LTI, LTV, and nonlinear, and propose an alternative approach to investigate total information rate via optimal estimation.

An Information-Theoretic Analysis of Discrete-Time Control and Filtering Limitations by the I-MMSE Relationships

TL;DR

This work extends the I-MMSE relationship to discrete-time AWGN channels with and without feedback and uses the resulting total information rate as a unifying metric for control and filtering trade-offs. By modeling control and filtering tasks as Gaussian channels and leveraging optimal estimation via causal/predictive MMSE, it derives sandwich bounds and exact equalities that connect information rate to unstable dynamics and Bode-type integrals, for LTI, LTV, and nonlinear systems. For control, it shows in the LTI case and in the LTV case, with nonlinear cases bounded via MMSE-based estimates; for filtering, it proves (LTV) or (LTI) under vanishing process noise, providing a lower bound on the best achievable MMSE. Collectively, the results offer an information-theoretic lens on time-averaged control and estimation limits and yield practical ways to bound and estimate fundamental trade-offs, with extensions to colored or non-Gaussian disturbances and to broader channel models as future work.

Abstract

Fundamental limitations or performance trade-offs/limits are important properties and constraints of both control and filtering systems. Among various trade-off metrics, total information rate that characterizes the sensitivity trade-offs and time-averaged performance of control and filtering systems was conventionally studied by using the differential entropy rate and Kolmogorov-Bode formula. In this paper, by extending the famous I-MMSE (mutual information -- minimum mean-square error) relationships to the discrete-time additive white Gaussian channels with and without feedback, a new paradigm is introduced to estimate and analyze total information rate as a control and filtering trade-off metric. Under this framework, we explore the trade-off properties of total information rate for a variety of the discrete-time control and filtering systems, e.g., LTI, LTV, and nonlinear, and propose an alternative approach to investigate total information rate via optimal estimation.
Paper Structure (15 sections, 18 theorems, 103 equations, 3 figures)

This paper contains 15 sections, 18 theorems, 103 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Preliminaries and Problem Formulation
  3. Information Theory
  4. Discrete-Time Gaussian Channel
  5. Discrete-Time Control Trade-offs
  6. General Control Systems and Trade-offs
  7. Control Trade-offs in LTI Systems
  8. Control Trade-offs in LTV Systems
  9. Control Trade-offs in Nonlinear Systems
  10. Discrete-Time Filtering Limits
  11. General Filtering Systems and Limits
  12. Filtering Limits in LTI Systems
  13. Filtering Limits in LTV Systems
  14. Filtering Limits in Nonlinear Systems
  15. Conclusion and Discussion

Key Result

Lemma 2.3

For a continuous-time additive Gaussian channel $dy^{}_t = \sqrt{\rm snr} \ \phi(t, m, y_0^t) dt + dw^{}_t$ with transmitted message $m$, signal-to-noise ratio ${\rm snr} > 0$, channel input process $\phi^{}_t(m, y_0^t)$ or $\phi^{}_t(m)$ satisfying $\mathbb{E}[\phi_t^\top \phi_t^{}] < \infty$, chan where ${\rm cmmse}(\phi_\tau, {\rm snr}) := \mathbb{E}[(\phi_\tau - \hat{\phi}_\tau)^\top (\phi_\t

Figures (3)

  • Figure 1: Discrete-time additive Gaussian channel.
  • Figure 2: Configuration of a general control system. (a) Block diagram of control system. (b) Block diagram of control input process $\mathcal{U}$.
  • Figure 3: Configuration of a general filtering system.

Theorems & Definitions (40)

  • Definition 2.1: Differential Entropy
  • Definition 2.2: Mutual Information
  • Lemma 2.3
  • Theorem 2.4
  • proof : Proof
  • Remark 3.1
  • Theorem 3.2
  • proof : Proof
  • Proposition 3.3
  • proof : Proof
  • ...and 30 more