Markovian Continuity of the MMSE

TL;DR

This work investigates when MMSE estimation is continuous with respect to standard stochastic convergences. It identifies that classical counterexamples arise from information leakage in the converging measurements and introduces Markovian convergence, defined via a Markov relation between the nominal parameter and the converging measurement, along with convergence of second moments. Under these conditions, it proves MMSE continuity in probability, and it further establishes upper-semicontinuity in distribution under weaker assumptions and full distributional continuity for common-parameter/degraded-channel settings; it also shows LMMSE continuity under joint second-moment convergence. The results unify and extend prior continuity guarantees, providing robustness assurances for MMSE and LMMSE in practical regimes with diminishing noise and finite-precision effects. Together, these findings offer principled guidance for reliable inference in signal processing and related fields.

Abstract

Minimum mean square error (MMSE) estimation is widely used in signal processing and related fields. While it is known to be non-continuous with respect to all standard notions of stochastic convergence, it remains robust in practical applications. In this work, we review the known counterexamples to the continuity of the MMSE. We observe that, in these counterexamples, the discontinuity arises from an element in the converging measurement sequence providing more information about the estimand than the limit of the measurement sequence. We argue that this behavior is uncharacteristic of real-world applications and introduce a new stochastic convergence notion, termed Markovian convergence, to address this issue. We prove that the MMSE is, in fact, continuous under this new notion. We supplement this result with semi-continuity and continuity guarantees of the MMSE in other settings and prove the continuity of the MMSE under linear estimation.

Figures (2)

• Figure 1: Illustration of a nested sequence of (physically) degraded channels $X \mathrel{\multimap}\joinrel\mathrel{-}\joinrel\mathrel{\mkern-6mu}\joinrel\mathrel{-} Y \mathrel{\multimap}\joinrel\mathrel{-}\joinrel\mathrel{\mkern-6mu}\joinrel\mathrel{-} \cdots \mathrel{\multimap}\joinrel\mathrel{-}\joinrel\mathrel{\mkern-6mu}\joinrel\mathrel{-} Y_{n+1} \mathrel{\multimap}\joinrel\mathrel{-}\joinrel\mathrel{\mkern-6mu}\joinrel\mathrel{-} Y_n \mathrel{\multimap}\joinrel\mathrel{-}\joinrel\mathrel{\mkern-6mu}\joinrel\mathrel{-} \cdots \mathrel{\multimap}\joinrel\mathrel{-}\joinrel\mathrel{\mkern-6mu}\joinrel\mathrel{-} Y_3 \mathrel{\multimap}\joinrel\mathrel{-}\joinrel\mathrel{\mkern-6mu}\joinrel\mathrel{-} Y_2 \mathrel{\multimap}\joinrel\mathrel{-}\joinrel\mathrel{\mkern-6mu}\joinrel\mathrel{-} Y_1$.
• Figure 2: Illustration of a sequence of individually (physically) degraded channels: $X \mathrel{\multimap}\joinrel\mathrel{-}\joinrel\mathrel{\mkern-6mu}\joinrel\mathrel{-} Y \mathrel{\multimap}\joinrel\mathrel{-}\joinrel\mathrel{\mkern-6mu}\joinrel\mathrel{-} Y_i$ for all $i \in \left\{ 1,2,\ldots,n \right\}$. This is a less stringent requirement than the one depicted in Figure \ref{['fig:nested-garbling']} as it does not assume degradedness between $X_i$ and $X_j$ for $i \neq j$.

Theorems & Definitions (46)

• Definition 2.1
• Theorem 2.1
• Example 2.1
• Example 2.2
• Example 2.3
• Theorem 2.2: Wu-Verdu:MMSE:IT2012
• Theorem 2.3: Wu-Verdu:MMSE:IT2012
• Remark 2.1
• Theorem 2.4: Hogeboom_thesis, Yuksel-Basar:games:book:2024
• Remark 2.2