Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Terminal Remote Generation and Estimation Over a Broadcast Channel With Correlated Priors

Maximilian Egger, Rawad Bitar, Antonia Wachter-Zeh, Nir Weinberger, Deniz Gündüz

TL;DR

This work tackles remote generation and estimation for a single encoder communicating with multiple decoders that observe correlated priors and must estimate functions of a target distribution $p_Q$ under a rate constraint. It introduces a hierarchical sampling scheme built on minimal random coding and importance sampling, using a broadcast link to exploit non-zero Gács-Körner common information to share a coarse block sample and then perform per-decoder refinements via unicast. The authors derive high-probability bias and estimation-error bounds for the point-to-point case and provide a complexity bound showing how communication cost scales with divergences such as $D_{KL}(p_{Q_C}\|p_C)$ and related quantities, highlighting gains when common information is present. The multi-terminal extension demonstrates that broadcasting a common sample across decoders can significantly reduce total communication, with potential extensions to more hierarchies and alternatives to GK common information. The results offer a pathway to efficient distributed generation and function estimation in ML contexts, where correlated priors and rate constraints are common.

Abstract

We study the multi-terminal remote estimation problem under a rate constraint, in which the goal of the encoder is to help each decoder estimate a function over a certain distribution -- while the distribution is known only to the encoder, the function to be estimated is known only to the decoders, and can also be different for each decoder. The decoders can observe correlated samples from prior distributions, instantiated through shared randomness with the encoder. To achieve this, we employ remote generation, where the encoder helps decoders generate samples from the underlying distribution by using the samples from the prior through importance sampling. While methods such as minimal random coding can be used to efficiently transmit samples to each decoder individually using their importance scores, it is unknown if the correlation among the samples from the priors can reduce the communication cost using the availability of a broadcast link. We propose a hierarchical importance sampling strategy that facilitates, in the case of non-zero Gács-Körner common information among the priors of the decoders, a common sampling step leveraging the availability of a broadcast channel. This is followed by a refinement step for the individual decoders. We present upper bounds on the bias and the estimation error for unicast transmission, which is of independent interest. We then introduce a method that splits into two phases, dedicated to broadcast and unicast transmission, respectively, and show the reduction in communication cost.

Multi-Terminal Remote Generation and Estimation Over a Broadcast Channel With Correlated Priors

TL;DR

This work tackles remote generation and estimation for a single encoder communicating with multiple decoders that observe correlated priors and must estimate functions of a target distribution under a rate constraint. It introduces a hierarchical sampling scheme built on minimal random coding and importance sampling, using a broadcast link to exploit non-zero Gács-Körner common information to share a coarse block sample and then perform per-decoder refinements via unicast. The authors derive high-probability bias and estimation-error bounds for the point-to-point case and provide a complexity bound showing how communication cost scales with divergences such as and related quantities, highlighting gains when common information is present. The multi-terminal extension demonstrates that broadcasting a common sample across decoders can significantly reduce total communication, with potential extensions to more hierarchies and alternatives to GK common information. The results offer a pathway to efficient distributed generation and function estimation in ML contexts, where correlated priors and rate constraints are common.

Abstract

We study the multi-terminal remote estimation problem under a rate constraint, in which the goal of the encoder is to help each decoder estimate a function over a certain distribution -- while the distribution is known only to the encoder, the function to be estimated is known only to the decoders, and can also be different for each decoder. The decoders can observe correlated samples from prior distributions, instantiated through shared randomness with the encoder. To achieve this, we employ remote generation, where the encoder helps decoders generate samples from the underlying distribution by using the samples from the prior through importance sampling. While methods such as minimal random coding can be used to efficiently transmit samples to each decoder individually using their importance scores, it is unknown if the correlation among the samples from the priors can reduce the communication cost using the availability of a broadcast link. We propose a hierarchical importance sampling strategy that facilitates, in the case of non-zero Gács-Körner common information among the priors of the decoders, a common sampling step leveraging the availability of a broadcast channel. This is followed by a refinement step for the individual decoders. We present upper bounds on the bias and the estimation error for unicast transmission, which is of independent interest. We then introduce a method that splits into two phases, dedicated to broadcast and unicast transmission, respectively, and show the reduction in communication cost.
Paper Structure (10 sections, 7 theorems, 14 equations, 2 figures)

This paper contains 10 sections, 7 theorems, 14 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Preliminaries and System Model
  3. Hierarchical Sampling
  4. Hierarchical Sampling Methodology
  5. Block Sampling
  6. Conditional Sampling
  7. On the Bias and Estimation Error
  8. Complexity Analysis
  9. Multi-Terminal Setting
  10. Conclusion

Key Result

Lemma 1

For some $t \geq 0$, let $n = \exp(D_{\text{KL}}( p_{\mathsf{Q}}\Vert p_{\mathsf{Y}_{i}}) + t)$ and $\epsilon \triangleq e^{-t/4} + 2 \sqrt{\Pr_{X \sim p_{\mathsf{Q}}}(\log(\rho_{i}^{}(X) > D_{\text{KL}}( p_{\mathsf{Q}}\Vert p_{\mathsf{Y}_{i}}) + t/2))})^{1/2}$, then with probability at least $1-2\ where the probability is over the randomness of $Y_{i, j}\sim p_{\mathsf{Y}_{i}}$.

Figures (2)

  • Figure 1: System model for two decoders.
  • Figure 2: Simple illustration of hierarchical sampling for two decoders with correlated priors $p_{\mathsf{Y}_{1}}$ and $p_{\mathsf{Y}_{2}}$ using Gács-Körner common information $\mathsf{C}$.

Theorems & Definitions (9)

  • Lemma 1: chatterjee2018sample
  • Proposition 1
  • Remark 1
  • Theorem 1
  • Corollary 1
  • Proposition 2
  • Corollary 2
  • Lemma 2
  • Remark 2