Adaptive Greedy Rejection Sampling

TL;DR

The paper addresses efficient one-shot channel simulation between two parties by coding a target distribution $Q$ with a shared proposal $P$. It generalizes greedy rejection sampling to adaptive, general-space settings (AGRS), proving correctness and codelength guarantees, and establishes that the non-adaptive GRS runtime is $\exp(D_{infty}(Q||P))$. It further reveals that Gaussian-channel instantiations can yield infinite expected runtime under standard schemes, motivating overdispersed proposals and a 1D AGRS variant with Bits-Back quantization to achieve finite runtime and favorable scaling. Empirical and theoretical results suggest that, in the Gaussian setting, a targeted AGRS construction may attain $O(D_{KL}(Q||P))$-type behavior, offering practical routes for efficient OSCS in continuous spaces.

Abstract

We consider channel simulation protocols between two communicating parties, Alice and Bob. First, Alice receives a target distribution $Q$, unknown to Bob. Then, she employs a shared coding distribution $P$ to send the minimum amount of information to Bob so that he can simulate a single sample $X \sim Q$. For discrete distributions, Harsha et al. (2009) developed a well-known channel simulation protocol -- greedy rejection sampling (GRS) -- with a bound of ${D_{KL}[Q \,\Vert\, P] + 2\ln(D_{KL}[Q \,\Vert\, P] + 1) + \mathcal{O}(1)}$ on the expected codelength of the protocol. In this paper, we extend the definition of GRS to general probability spaces and allow it to adapt its proposal distribution after each step. We call this new procedure Adaptive GRS (AGRS) and prove its correctness. Furthermore, we prove the surprising result that the expected runtime of GRS is exactly $\exp(D_\infty[Q \,\Vert\, P])$, where $D_\infty[Q \,\Vert\, P]$ denotes the Rényi $\infty$-divergence. We then apply AGRS to Gaussian channel simulation problems. We show that the expected runtime of GRS is infinite when averaged over target distributions and propose a solution that trades off a slight increase in the coding cost for a finite runtime. Finally, we describe a specific instance of AGRS for 1D Gaussian channels inspired by hybrid coding. We conjecture and demonstrate empirically that the runtime of AGRS is $\mathcal{O}(D_{KL}[Q \,\Vert\, P])$ in this case.

Figures (3)

• Figure 1:
• Figure 2:
• Figure 3: Bits-back Quantization encoding and decoding procedure for a target distribution of $Q = \mathcal{U}(\kappa + [0, 3/4))$, i.e., $a = 3$ and $b = 4$. In the figure, $\mathop{\mathrm{supp}}\nolimits Y_I$ and $\mathop{\mathrm{supp}}\nolimits \kappa_I$ denote the support of $Y_I$ and $\kappa_I$, respectively. The encoding procedure begins in the top left with "Alice's initial message $m$" and follows the solid arrows. For a detailed description of the procedure and the notation, see \ref{['sec:bbq']}. We assume that Alice already has some message $m$ she wishes to communicate to Bob. We also assume that both Alice and Bob can simulate $V$ with the publicly available randomness $\mathcal{BP}$ and they use the same invertible sampler to encode and decode messages to/from $m$. Crucially, since Alice begins by decoding $\log_2 3$ bits from her message, at the end of the encoding process her message length only increases by $-\log_2 3/4$ bits in total.

Theorems & Definitions (14)

• Theorem 3.1
• proof
• Lemma 3.1
• proof
• Lemma 3.2
• proof
• Lemma 3.3
• proof
• Theorem 3.2
• proof