Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A New Finite-Horizon Dynamic Programming Analysis of Nonanticipative Rate-Distortion Function for Markov Sources

Zixuan He, Charalambos D. Charalambous, Photios A. Stavrou

TL;DR

This paper derived a new information structure for the NRDF and new convexity results that allow reformulating the problem as an unconstrained partially observable finite-horizon stochastic dynamic program (DP) using Lagrange duality theorem and proposes a novel alternating minimization scheme to approximate both the control policy and cost-to-go function through backward recursions with provable convergence guarantees.

Abstract

This paper deals with the computation of a non-asymptotic lower bound by means of the nonanticipative rate-distortion function (NRDF) on the discrete-time zero-delay variable-rate lossy compression problem for discrete Markov sources with per-stage, single-letter distortion. First, we derive a new information structure of the NRDF for Markov sources and single-letter distortions. Second, we derive new convexity results on the NRDF, which facilitate the use of Lagrange duality theorem to cast the problem as an unconstrained partially observable finite-time horizon stochastic dynamic programming (DP) algorithm subject to a probabilistic state (belief state) that summarizes the past information about the reproduction symbols and takes values in a continuous state space. Instead of approximating the DP algorithm directly, we use Karush-Kuhn-Tucker (KKT) conditions to find an implicit closed-form expression of the optimal control policy of the stochastic DP (i.e., the minimizing distribution of the NRDF) and approximate the control policy and the cost-to-go function (a function of the rate) stage-wise, via a novel dynamic alternating minimization (AM) approach, that is realized by an offline algorithm operating using backward recursions, with provable convergence guarantees. We obtain the clean values of the aforementioned quantities using an online (forward) algorithm operating for any finite-time horizon. Our methodology provides an approximate solution to the exact NRDF solution, which becomes near-optimal as the search space of the belief state becomes sufficiently large at each time stage. We corroborate our theoretical findings with simulation studies where we apply our algorithms assuming time-varying and time-invariant binary Markov processes.

A New Finite-Horizon Dynamic Programming Analysis of Nonanticipative Rate-Distortion Function for Markov Sources

TL;DR

This paper derived a new information structure for the NRDF and new convexity results that allow reformulating the problem as an unconstrained partially observable finite-horizon stochastic dynamic program (DP) using Lagrange duality theorem and proposes a novel alternating minimization scheme to approximate both the control policy and cost-to-go function through backward recursions with provable convergence guarantees.

Abstract

This paper deals with the computation of a non-asymptotic lower bound by means of the nonanticipative rate-distortion function (NRDF) on the discrete-time zero-delay variable-rate lossy compression problem for discrete Markov sources with per-stage, single-letter distortion. First, we derive a new information structure of the NRDF for Markov sources and single-letter distortions. Second, we derive new convexity results on the NRDF, which facilitate the use of Lagrange duality theorem to cast the problem as an unconstrained partially observable finite-time horizon stochastic dynamic programming (DP) algorithm subject to a probabilistic state (belief state) that summarizes the past information about the reproduction symbols and takes values in a continuous state space. Instead of approximating the DP algorithm directly, we use Karush-Kuhn-Tucker (KKT) conditions to find an implicit closed-form expression of the optimal control policy of the stochastic DP (i.e., the minimizing distribution of the NRDF) and approximate the control policy and the cost-to-go function (a function of the rate) stage-wise, via a novel dynamic alternating minimization (AM) approach, that is realized by an offline algorithm operating using backward recursions, with provable convergence guarantees. We obtain the clean values of the aforementioned quantities using an online (forward) algorithm operating for any finite-time horizon. Our methodology provides an approximate solution to the exact NRDF solution, which becomes near-optimal as the search space of the belief state becomes sufficiently large at each time stage. We corroborate our theoretical findings with simulation studies where we apply our algorithms assuming time-varying and time-invariant binary Markov processes.

Paper Structure

This paper contains 5 sections, 6 theorems, 31 equations, 3 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Problem Statement and Preliminaries
  3. Main Results
  4. Numerical Examples
  5. Conclusion

Key Result

Lemma 1

(Structural Property) For a given Markov source $\{P_t(x_t|x_{t-1}):~t\in\mathbb{N}_0^n\}$ and a single letter distortion function $\{\rho_t(x_t,y_t):~t\in\mathbb{N}_0^n\}$, the characterization in NRDF can be simplified as followsFor non-empty finite sets, we can replace infimum with minimum due to where

Figures (3)

  • Figure 1: A generic zero-delay lossy source coding system
  • Figure 2: Illustration of the rate & distortion per stage and stage-wise convergence for the time-varying case.
  • Figure 3: Illustration of the best trajectory $\{P_t^*(x_{t-1}|y_{t-1}):t\in\mathbb{N}_1^n\}$, the corresponding test-channels, output, and rate per stage ($t\in\mathbb{N}_0^n$) in the time-invariant case.

Theorems & Definitions (15)

  • Remark 1
  • Lemma 1
  • proof
  • Theorem 2
  • proof
  • Theorem 3
  • proof
  • Lemma 4
  • proof
  • Theorem 5
  • ...and 5 more