Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Derivation of Mutual Information and Linear Minimum Mean-Square Error for Viterbi Decoding of Convolutional Codes Using the Innovations Method

Masato Tajima

TL;DR

An upper bound on the average mutual information per branch for Viterbi decoding of convolutional codes is given using these covariance matrices derived using the formula in the Kalman filter.

Abstract

We see that convolutional coding/Viterbi decoding has the structure of the Kalman filter (or the linear minimum variance filter). First, we calculate the covariance matrix of the innovation (i.e., the soft-decision input to the main decoder in a Scarce-State-Transition (SST) Viterbi decoder). Then a covariance matrix corresponding to that of the one-step prediction error in the Kalman filter is obtained. Furthermore, from that matrix, a covariance matrix corresponding to that of the filtering error in the Kalman filter is derived using the formula in the Kalman filter. As a result, the average mutual information per branch for Viterbi decoding of convolutional codes is given using these covariance matrices. Also, the trace of the latter matrix represents the linear minimum mean-square error (LMMSE). We show that an approximate value of the average mutual information is sandwiched between half the SNR times the average filtering and one-step prediction LMMSEs. In the case of QLI codes, from the covariance matrix of the soft-decision input to the main decoder, we can get a matrix. We show that the trace of this matrix has some connection with the linear smoothing error.

Derivation of Mutual Information and Linear Minimum Mean-Square Error for Viterbi Decoding of Convolutional Codes Using the Innovations Method

TL;DR

An upper bound on the average mutual information per branch for Viterbi decoding of convolutional codes is given using these covariance matrices derived using the formula in the Kalman filter.

Abstract

We see that convolutional coding/Viterbi decoding has the structure of the Kalman filter (or the linear minimum variance filter). First, we calculate the covariance matrix of the innovation (i.e., the soft-decision input to the main decoder in a Scarce-State-Transition (SST) Viterbi decoder). Then a covariance matrix corresponding to that of the one-step prediction error in the Kalman filter is obtained. Furthermore, from that matrix, a covariance matrix corresponding to that of the filtering error in the Kalman filter is derived using the formula in the Kalman filter. As a result, the average mutual information per branch for Viterbi decoding of convolutional codes is given using these covariance matrices. Also, the trace of the latter matrix represents the linear minimum mean-square error (LMMSE). We show that an approximate value of the average mutual information is sandwiched between half the SNR times the average filtering and one-step prediction LMMSEs. In the case of QLI codes, from the covariance matrix of the soft-decision input to the main decoder, we can get a matrix. We show that the trace of this matrix has some connection with the linear smoothing error.
Paper Structure (28 sections, 24 theorems, 273 equations, 6 figures, 10 tables)

This paper contains 28 sections, 24 theorems, 273 equations, 6 figures, 10 tables.

Table of Contents

  1. Introduction and Scenario
  2. Kalman Filter
  3. The Discrete-Time Kalman Filter
  4. Linear Minimum Variance Filter
  5. Mutual Information Associated with the Discrete-Time Kalman Filter
  6. Gaussian Signals
  7. Non-Gaussian Signals
  8. Relationship Between Mutual Information and LMMSE
  9. Innovations for Viterbi Decoding of Convolutional Codes
  10. Covariance Matrix of the Innovation
  11. Mutual Information and LMMSE for Viterbi Decoding of Convolutional Codes
  12. Mutual Information and LMMSE in the Case of $n_0=2$
  13. Numerical Results
  14. Discussion
  15. $\frac{1}{2 \rho}\log \vert I_2+\rho \Sigma_x \vert \leq \frac{\log(1+\rho)}{\rho}$
  16. ...and 13 more sections

Key Result

Proposition 1

The covariance matrix of the innovation $\hbox{\boldmath $\nu$}_k$ is given by where $S_k$ is the covariance matrix of the error in the estimate, i.e., $\hbox{\boldmath $s$}_k-\hbox{\boldmath $\hat{s}$}_{k \vert k-1}$.

Figures (6)

  • Figure 1: The structure of an SST Viterbi decoder (pre-decoder: $G^{-1}$).
  • Figure 2: Relationship between $\frac{1}{2}\hbox{tr}(\Sigma_c)$, $\frac{1}{2 \rho}\log \vert I_2+\rho \Sigma_x \vert$, and $\frac{1}{2}\hbox{tr}(\Sigma_x)$ ($C_1$).
  • Figure 3: Relationship between $\frac{1}{2}\hbox{tr}(\Sigma_c)$, $\frac{1}{2 \rho}\log \vert I_2+\rho \Sigma_x \vert$, and $\frac{1}{2}\hbox{tr}(\Sigma_x)$ ($C_2$).
  • Figure 4: Relationship between $\frac{1}{2 \rho}\log \vert I_2+\rho \Sigma_x \vert$, $\frac{2I(\rho)}{\rho}$, and $\frac{\log(1+\rho)}{\rho}$ ($C_1$).
  • Figure 5: Relationship between $\frac{1}{2 \rho}\log \vert I_2+\rho \Sigma_x \vert$, $\frac{2I(\rho)}{\rho}$, and $\frac{\log(1+\rho)}{\rho}$ ($C_2$).
  • ...and 1 more figures

Theorems & Definitions (24)

  • Proposition 1: Kailath kai 681
  • Proposition 2: Arimoto ari 77
  • Proposition 3: Kailath kai 681
  • Proposition 4: Arimoto ari 77
  • Proposition 5
  • Proposition 6
  • Proposition 7
  • Lemma 1
  • Proposition 8
  • Lemma 2
  • ...and 14 more