Rate-Distortion Region for Distributed Indirect Source Coding with Decoder Side Information

TL;DR

This work tackles distributed indirect rate-distortion with decoder side information, where $M$ encoders independently compress observations to aid a central decoder that has access to correlated side information $Y$ and must reconstruct a latent variable $T$ under distortion $D$. It proves an exact single-letter rate-distortion characterization when the sources are conditionally independent given $Y$, by showing inner and outer regions coincide (i.e., $R_a(D)=R_o(D)=R^{*}_{X_1, dots,X_M|Y}(D)$) and providing a simplifying corollary $R_i \ge I(X_i; W_i) - I(W_i; Y)$. For computation, it develops a distributed Blahut–Arimoto algorithm with auxiliary distributions $Q_i$, $q_i$, and $q'$, using an alternating-minimization scheme to obtain practical inner bounds on the rate-distortion region. Numerical examples with two binary sources validate the method and align with the Wyner–Ziv result in the single-source limit, illustrating the framework’s potential for task-oriented semantic communication and distributed learning applications.

Abstract

This paper studies a variant of the rate-distortion problem motivated by task-oriented semantic communication and distributed learning systems, where $M$ correlated sources are independently encoded for a central decoder. The decoder has access to correlated side information in addition to the messages received from the encoders and aims to recover a latent random variable under a given distortion constraint, rather than recovering the sources themselves. We characterize the exact rate-distortion function for the case where the sources are conditionally independent given the side information. Furthermore, we develop a distributed Blahut-Arimoto (BA) algorithm to numerically compute the rate-distortion function. Numerical examples are provided to demonstrate the effectiveness of the proposed approach in calculating the rate-distortion region.

Figures (4)

• Figure 1: Distributed remote compression of a latent variable with $M$ correlated sources at distributed transmitters and side information at the receiver.
• Figure 2: Contour plot of the rate distortion region with two distributed binary sources $\{X_1, X_2\}$, where the labels on the contours represent the distortion values $D$ on $d(t, \hat{t})$.
• Figure 3: Surface plot of the rate-distortion region.
• Figure 4: The rate-distortion function for the case when $M=1$, i.e., the Wyner-Ziv problem