Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Information-Theoretic Analysis of Continuous-Time Control and Filtering Limitations by the I-MMSE Relationships

Neng Wan, Dapeng Li, Naira Hovakimyan

TL;DR

This work introduces a unified, continuous-time information-theoretic framework for control and filtering by embedding them into additive Gaussian channels and applying the I-MMSE relationships. A direct method based on Duncan's theorem lifts the total information rate to infinite-dimensional function spaces and expresses it via causal MMSE estimators, enabling exact equalities in linear settings and meaningful bounds in nonlinear cases. For continuous-time LTI plants, the authors prove an equality between the control-information rate and the sum of open-loop unstable poles; for LTV plants they derive a time-domain, Bode-like integral that bounds the rate in terms of the antistable spectrum; for nonlinear systems, Stratonovich-Kushner-based estimations provide computable lower bounds. In filtering, analogous results show that the information rate corresponds to half of the minimum mean-squared error, with Kalman-Bucy filtering emerging as a special case under vanishing process noise. Overall, the framework unifies and extends classical limits (Bode-type integrals, data-rate theorems, entropy costs) to general continuous-time control and filtering systems without forcing stationarity or Gaussianity assumptions, offering practical tools for analyzing fundamental performance limits.

Abstract

While information theory has been introduced to characterize the fundamental limitations of control and filtering for a few decades, the existing information-theoretic methods are indirect and cumbersome for analyzing the limitations of continuous-time systems. To answer this challenge, we lift the information-theoretic analysis to continuous function spaces by the I-MMSE relationships. Continuous-time control and filtering systems are modeled into the additive Gaussian channels with and without feedback, and the total information rate is identified as a control and filtering trade-off metric and calculated from the estimation error of channel inputs. Fundamental constraints for this trade-off metric are first derived in a general setup and then used to capture the limitations of various control and filtering systems subject to linear and nonlinear plant models. For linear scenarios, we show that the total information rate quantifies the performance limits, such as the minimum entropy cost and the lowest achievable mean-square estimation error, in the time domain. For nonlinear systems, we provide a direct method to calculate and interpret the total information rate and its lower bound by the Stratonovich-Kushner equation.

An Information-Theoretic Analysis of Continuous-Time Control and Filtering Limitations by the I-MMSE Relationships

TL;DR

This work introduces a unified, continuous-time information-theoretic framework for control and filtering by embedding them into additive Gaussian channels and applying the I-MMSE relationships. A direct method based on Duncan's theorem lifts the total information rate to infinite-dimensional function spaces and expresses it via causal MMSE estimators, enabling exact equalities in linear settings and meaningful bounds in nonlinear cases. For continuous-time LTI plants, the authors prove an equality between the control-information rate and the sum of open-loop unstable poles; for LTV plants they derive a time-domain, Bode-like integral that bounds the rate in terms of the antistable spectrum; for nonlinear systems, Stratonovich-Kushner-based estimations provide computable lower bounds. In filtering, analogous results show that the information rate corresponds to half of the minimum mean-squared error, with Kalman-Bucy filtering emerging as a special case under vanishing process noise. Overall, the framework unifies and extends classical limits (Bode-type integrals, data-rate theorems, entropy costs) to general continuous-time control and filtering systems without forcing stationarity or Gaussianity assumptions, offering practical tools for analyzing fundamental performance limits.

Abstract

While information theory has been introduced to characterize the fundamental limitations of control and filtering for a few decades, the existing information-theoretic methods are indirect and cumbersome for analyzing the limitations of continuous-time systems. To answer this challenge, we lift the information-theoretic analysis to continuous function spaces by the I-MMSE relationships. Continuous-time control and filtering systems are modeled into the additive Gaussian channels with and without feedback, and the total information rate is identified as a control and filtering trade-off metric and calculated from the estimation error of channel inputs. Fundamental constraints for this trade-off metric are first derived in a general setup and then used to capture the limitations of various control and filtering systems subject to linear and nonlinear plant models. For linear scenarios, we show that the total information rate quantifies the performance limits, such as the minimum entropy cost and the lowest achievable mean-square estimation error, in the time domain. For nonlinear systems, we provide a direct method to calculate and interpret the total information rate and its lower bound by the Stratonovich-Kushner equation.
Paper Structure (17 sections, 19 theorems, 105 equations, 2 figures)

This paper contains 17 sections, 19 theorems, 105 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Preliminaries and Problem Formulation
  3. Probability in Function Spaces
  4. Information Theory
  5. Continuous-Time Gaussian Channel Model
  6. Continuous-Time Control Systems
  7. Continuous-Time Filtering Systems
  8. Fundamental Limitations of Control
  9. General Control Trade-offs
  10. Control Trade-offs in LTI Systems
  11. Control Trade-offs in LTV Systems
  12. Control Trade-offs in Nonlinear Systems
  13. Fundamental Limitations of Filtering
  14. General Filtering Limits
  15. Filtering Limits in LTI Systems
  16. ...and 2 more sections

Key Result

Lemma 2.6

For the Gaussian channel Gaussian_Channel subject to (ass1)-(ass3), the mutual information $I(y_0^t; m)$ between the channel output $y_0^t$ and message $m$, is given by where $\hat{\phi}(\tau, y_0^\tau, m) = \mathbb{E}\left[ \phi(\tau, y_0^\tau, m) | y_0^\tau \right]$.For a function $f(\tau, y_0^\tau, m) \in \mathcal{L}^2([0, t])$, $\|f\|^2 = \|f\|^2_{(t, \omega)} := \intop_{0}^{t}f^\top(\tau, y

Figures (2)

  • Figure 1: Control configuration. (a) Block diagram of control systems. (b) Lumped control process $\mathcal{U}$.
  • Figure 2: Filtering configuration.

Theorems & Definitions (34)

  • Definition 2.1
  • Definition 2.2
  • Definition 2.3
  • Remark 2.4
  • Remark 2.5
  • Lemma 2.6
  • Lemma 2.7
  • Remark 2.8
  • Theorem 3.1
  • Proposition 3.2
  • ...and 24 more