Computation of a Unified Graph-Based Rate Optimization Problem

TL;DR

The work introduces a unified graph-based framework to compute fundamental limits in source and channel coding with side information, unifying graph entropy, rate-distortion, and capacity-cost concepts under the objective $I(U;V)-I(U;W)$. It provides graph-characterization results, a flexible alternating minimization algorithm with adaptive multiplier updates, and deflation plus graph-simplification techniques to dramatically reduce computational cost. The authors prove convergence to optimal solutions and demonstrate substantial speedups over traditional Blahut–Arimoto-type methods, enabling computation on large problem instances. Numerical experiments across RDF, capacity-cost, and Gaussian-state channels with quantized information validate accuracy and efficiency, highlighting practical impact for complex rate-loss tradeoffs with side information.

Abstract

We define a graph-based rate optimization problem and consider its computation, which provides a unified approach to the computation of various theoretical limits, including the (conditional) graph entropy, rate-distortion functions and capacity-cost functions with side information. Compared with their classical counterparts, theoretical limits with side information are much more difficult to compute since their characterizations as optimization problems have larger and more complex feasible regions. Following the unified approach, we develop effective methods to resolve the difficulty. On the theoretical side, we derive graph characterizations for rate-distortion and capacity-cost functions with side information and simplify the characterizations in special cases by reducing the number of decision variables. On the computational side, we design an efficient alternating minimization algorithm for the graph-based problem, which deals with the inequality constraint by a flexible multiplier update strategy. Moreover, simplified graph characterizations are exploited and deflation techniques are introduced, so that the computing time is greatly reduced. Theoretical analysis shows that the algorithm converges to an optimal solution. By numerical experiments, the accuracy and efficiency of the algorithm are illustrated and its significant advantage over existing methods is demonstrated.

Figures (6)

• Figure 1: The rate-loss curve $T(L)$, compared with its special cases, the rate-distortion curve $R(D)$ in \ref{['eg:rdf']} (or the rate-distortion curve in \ref{['eg:crdf']}) and the capacity-cost curve $C(B)$ in \ref{['eg:ccf']}.
• Figure 2: Analytical (superscript $A$) and Numerical results (superscript $C$) for the first (upper) and second (lower) examples in \ref{['subsec:classical']}. In each case, we compute the optimal rate (capacity) with $150$ iterations for each point.
• Figure 3: Numerical results for the first (upper) and second (lower) examples in \ref{['subsubsec:rdf']}. In each case, we choose $50$ consecutive points from the intervals uniformly and compute the corresponding optimal rate with $1000$ iterations.
• Figure 4: Numerical results for the channel problem with quantized side information in \ref{['subsubsec:Gaussian']}. Capacity-cost curves for two schemes with different quantization granularity are plotted. In each case, $50$ consecutive points are chosen from the intervals uniformly and the corresponding capacity is computed with $1000$ iterations. The capacity-cost curve of the writing on dirty scheme is also plotted as an upper bound.
• Figure : Flexible Alternating Minimization Algorithm

Theorems & Definitions (46)

• Lemma 1
• Remark 1
• Remark 2
• Remark 3
• Lemma 2
• Remark 4
• Lemma 3
• Lemma 4
• Lemma 5
• Lemma 6