Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Capacity-Maximizing Input Symbol Selection for Discrete Memoryless Channels

Maximilian Egger, Rawad Bitar, Antonia Wachter-Zeh, Deniz Gündüz, Nir Weinberger

TL;DR

This work tackles capacity-maximizing input-symbol selection for discrete memoryless channels by seeking a subset of input symbols of size at most $k$ that maximizes the channel capacity $C(W_\mathcal{R})$. It shows that the problem is neither concave nor submodular, motivates a bound-guided approach, and develops theoretical guarantees based on the convex hull of conditional distributions. A clustering-based algorithm is proposed to select $k$ representatives by grouping similar input rows and maximizing hull coverage, with a bound that informs representative choice. Empirical results on Dirichlet-generated DMCs indicate the method closely tracks optimal selections and provides practical capacity-loss guarantees for large alphabets, offering a scalable alternative to exhaustive search.

Abstract

Motivated by communication systems with constrained complexity, we consider the problem of input symbol selection for discrete memoryless channels (DMCs). Given a DMC, the goal is to find a subset of its input alphabet, so that the optimal input distribution that is only supported on these symbols maximizes the capacity among all other subsets of the same size (or smaller). We observe that the resulting optimization problem is non-concave and non-submodular, and so generic methods for such cases do not have theoretical guarantees. We derive an analytical upper bound on the capacity loss when selecting a subset of input symbols based only on the properties of the transition matrix of the channel. We propose a selection algorithm that is based on input-symbols clustering, and an appropriate choice of representatives for each cluster, which uses the theoretical bound as a surrogate objective function. We provide numerical experiments to support the findings.

Capacity-Maximizing Input Symbol Selection for Discrete Memoryless Channels

TL;DR

This work tackles capacity-maximizing input-symbol selection for discrete memoryless channels by seeking a subset of input symbols of size at most that maximizes the channel capacity . It shows that the problem is neither concave nor submodular, motivates a bound-guided approach, and develops theoretical guarantees based on the convex hull of conditional distributions. A clustering-based algorithm is proposed to select representatives by grouping similar input rows and maximizing hull coverage, with a bound that informs representative choice. Empirical results on Dirichlet-generated DMCs indicate the method closely tracks optimal selections and provides practical capacity-loss guarantees for large alphabets, offering a scalable alternative to exhaustive search.

Abstract

Motivated by communication systems with constrained complexity, we consider the problem of input symbol selection for discrete memoryless channels (DMCs). Given a DMC, the goal is to find a subset of its input alphabet, so that the optimal input distribution that is only supported on these symbols maximizes the capacity among all other subsets of the same size (or smaller). We observe that the resulting optimization problem is non-concave and non-submodular, and so generic methods for such cases do not have theoretical guarantees. We derive an analytical upper bound on the capacity loss when selecting a subset of input symbols based only on the properties of the transition matrix of the channel. We propose a selection algorithm that is based on input-symbols clustering, and an appropriate choice of representatives for each cluster, which uses the theoretical bound as a surrogate objective function. We provide numerical experiments to support the findings.
Paper Structure (12 sections, 4 theorems, 31 equations, 1 figure, 1 algorithm)

This paper contains 12 sections, 4 theorems, 31 equations, 1 figure, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Theoretical Guarantees on Input Symbol Selection
  4. Symbol Selection Without Capacity Loss
  5. Bounding the Capacity Loss
  6. Input Symbol Selection by Clustering
  7. Conclusion
  8. Counter-Examples for Submodularity of a DMC
  9. Proof of \ref{['thm:inconvexhull']}
  10. Proof of \ref{['lemma:pseudo_capacity_diff']}
  11. Proof of \ref{['thm:closetohull']}
  12. Proof of \ref{['lemma:pseudo_capacity_diff_hull']}

Key Result

Proposition 1

Consider a channel $W = \{W_{\mathrm{Y}|\mathrm{X}=x}\}_{x \in \mathcal{X}}$, where the input symbols $\mathcal{X}$ are partitioned into two disjoint sets $\mathcal{U}$ and $\mathcal{R}$, such that the conditional distributions $W_{\mathrm{Y}|\mathrm{X}=u}$ of the symbols in $u \in \mathcal{U}$ are Hence, there exists a capacity-achieving input distribution for which $P_\mathrm{X}(x)=0$ for $x \i

Figures (1)

  • Figure 1: Input selection results for $50$ DMCs with input and output alphabet size $\vert \mathcal{X} \vert = \vert \mathcal{Y} \vert = 30$, randomly sampled with $\nu=5$, $d_1 = 0.005$ and $d_2=10^{10}$. Lines show average results, shaded areas the standard deviation.

Theorems & Definitions (15)

  • Definition 1: Submodular Set Functions
  • Definition 2: Convex Hull of a Channel
  • Proposition 1
  • Definition 3: Nearest Neighbor
  • Definition 4: Distance to the Convex Hull of a Channel
  • Definition 5: Pseudo-Simplex and Pseudo-Capacity
  • Theorem 1
  • proof : Sketch of the Proof
  • Lemma 1
  • proof
  • ...and 5 more