Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Phase Transitions of the Additive Uniform Noise Channel with Peak Amplitude and Cost Constraint

Jonas Stapmanns, Catarina Dias, Luke Eilers, Tobias Kühn, Jean-Pascal Pfister

TL;DR

The paper analyzes when the capacity-achieving input for an additive uniform-noise channel with peak-amplitude and cost constraints is discrete versus continuous. It derives a rigorous, unique optimal distribution via Smith’s optimality conditions and a Lagrangian framework, and then characterizes the phase transitions as the cost constraint becomes active or as the cost function curvature changes. Specifically, a concave cost leads to discrete optimal inputs, while a strictly convex cost induces a fully supported input on [0,1], with discrete solutions still possible under certain regimes; the capacity in each regime is given by explicit formulas, including mixtures of discrete entropies and, for the convex-case, differential entropies of the output. These results illuminate how constraint geometry governs input discreteness and offer analytical insight into capacity under realistic amplitude and resource budgets, with potential extensions to broader noise models and soft constraints.

Abstract

Under which condition is quantization optimal? We address this question in the context of the additive uniform noise channel under peak amplitude and cost constraints. We compute analytically the capacity-achieving input distribution as a function of the noise level, the average cost constraint, and the curvature of the cost function. We find that when the cost function is concave, the capacity-achieving input distribution is discrete, whereas when the cost function is convex and the cost constraint is active, the support of the capacity-achieving input distribution spans the entire interval. For the cases of a discrete capacity-achieving input distribution, we derive the analytical expressions for the capacity of the channel.

Phase Transitions of the Additive Uniform Noise Channel with Peak Amplitude and Cost Constraint

TL;DR

The paper analyzes when the capacity-achieving input for an additive uniform-noise channel with peak-amplitude and cost constraints is discrete versus continuous. It derives a rigorous, unique optimal distribution via Smith’s optimality conditions and a Lagrangian framework, and then characterizes the phase transitions as the cost constraint becomes active or as the cost function curvature changes. Specifically, a concave cost leads to discrete optimal inputs, while a strictly convex cost induces a fully supported input on [0,1], with discrete solutions still possible under certain regimes; the capacity in each regime is given by explicit formulas, including mixtures of discrete entropies and, for the convex-case, differential entropies of the output. These results illuminate how constraint geometry governs input discreteness and offer analytical insight into capacity under realistic amplitude and resource budgets, with potential extensions to broader noise models and soft constraints.

Abstract

Under which condition is quantization optimal? We address this question in the context of the additive uniform noise channel under peak amplitude and cost constraints. We compute analytically the capacity-achieving input distribution as a function of the noise level, the average cost constraint, and the curvature of the cost function. We find that when the cost function is concave, the capacity-achieving input distribution is discrete, whereas when the cost function is convex and the cost constraint is active, the support of the capacity-achieving input distribution spans the entire interval. For the cases of a discrete capacity-achieving input distribution, we derive the analytical expressions for the capacity of the channel.
Paper Structure (16 sections, 16 theorems, 82 equations, 8 figures)

This paper contains 16 sections, 16 theorems, 82 equations, 8 figures.

Table of Contents

  1. Introduction
  2. Problem statement
  3. results
  4. Capacity-achieving input distribution
  5. Capacity
  6. Discussion
  7. Proof of the uniqueness of the solution to the optimization problem \ref{['eq:px_max_inf_problem']}
  8. Proof of \ref{['lem:convexity_px']} (Strict concavity of mutual information in $p_X$)
  9. Proof of \ref{['lem:chain_non_overlapping']} (Uniqueness of $\lambda^{\ast}$)
  10. Proof of Lemma \ref{['lem:linear_i']} (piecewise linearity of the marginal information density)
  11. Proof of Case II$\mathrm{b}$
  12. Prerequisites
  13. Proof of Case IIb
  14. Step A
  15. Step B
  16. ...and 1 more sections

Key Result

Lemma 3

Let $p_{X,\left(\lambda,\nu\right)}^{\ast}\left(x\right)$ be the solution to the problem $\underset{p_X}{\max}\, {\cal L}_{}\left[p_X,\nu,\lambda\right]$ for a fixed $\left(\nu,\lambda\right)$. Then

Figures (8)

  • Figure 1: The different cases discussed in Theorem \ref{['thm:main']}. In the left column $r\in\mathbb{N}$ ($r=4$) and in the right column $r\notin\mathbb{N}$ ($r=4.4$). Top: Phase diagram in the $\alpha$-$\bar{c}$-plane. Green and red background indicate $p_X^{\ast}$ with discrete support and support on the entire interval $[0,1]$, respectively. Ia,b and IIa,b: discrete $p_X^{\ast}$ with masses $m_j$ and positions $x_j$ indicated by the heights and the positions of the blue arrows (dark blue for j even). The corresponding $p_{N}(y-x_j)$ is illustrated by dashed boxes in Ia/IIa and by dotted (j odd) and dashed (j even) boxes in Ib/IIb. The black line is the resulting $p_Y^{\ast}$. IIIa and IIIb: numerical result for $p_X^{\ast}$ (blue) using the Blahut-Arimoto algorithm blahut_1972arimoto_1972 and corresponding $p_Y^{\ast}$ in black.
  • Figure 2: Illustration of the the positions and masses belonging to $S_{k}^{<}$, $M_{k}^{<}$, $S_{k}^{>}$, and $M_{k}^{>}$. Capacity-achieving input distribution $p_X^{\ast}$ (blue arrows) and corresponding output distribution (black solid curve) for parameter values $r=6.2$, $\alpha=0.5$. The masses $m_2=m_4=m_6=0$ and $m_8>0$, so that the support is given by $S_k$ with $k=3$.
  • Figure 3: a) Masses $m_j$, $j=1,\,\dots,\,N_r$, as a function of $\bar{c}$. b) The Lagrange multiplier $\lambda^{\ast}$ as a function of $\bar{c}$. The values $\bar{c}=\theta_k$ (dashed vertical lines) denote the cost budget at which $m_{2k+2}$ vanishes and the support changes from $S_k$ to $S_{k+1}$ as the cost budget tightens. The corresponding values of the Lagrange multiplier are depicted by horizontal dashed lines. Here, $\alpha=0.5$ and $r=3.9$.
  • Figure 4: Illustration of the computation of the marginal information density. In case of the uniform noise with density $p_{N}\left(y - x\right)$, the marginal information density evaluated at $x=x^{\prime}$ is given by the convolution of $p_{N}\left(y - x^{\prime}\right)$ (blue dashed box) with the output probability density $p_Y\left(y\right)$ (black solid curve).
  • Figure 5: a) The r.h.s. and the l.h.s. of (\ref{['eq:ineq_constr']}), illustrating the linear interpolation between the points of support, where (\ref{['eq:eq_constr']}) ensures equality. Other parameters: $r=2.4$ and $\bar{c}=0.54<\bar{c}^\ast$. b) $p_X^\ast\left(x\right)$ as a function of $\alpha$ obtained numerically by means of the Blahut-Arimoto algorithm blahut_1972arimoto_1972, and ch. 7.2 of dauwels_2006. For $\alpha\leq1$, $p_X$ is discrete and for $\alpha>1$, it has support on the entire interval $[0,1]$. Other parameters: $r=2.4$ and $\bar{c}=0.35<\bar{c}^\ast$.
  • ...and 3 more figures

Theorems & Definitions (30)

  • Definition 2
  • Lemma 3: Strict concavity of mutual information in $p_X$
  • Lemma 4: Uniqueness of $\lambda^{\ast}$
  • Theorem 5: Optimality conditions; Smith, smith_information_1971
  • Definition 6
  • Definition 8
  • Remark 9
  • Definition 10
  • Theorem 11: Main Theorem
  • Remark 12
  • ...and 20 more