Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Class of Subadditive Information Measures and their Applications

Hamidreza Abin, Mahdi Zinati, Amin Gohari, Mohammad Hossein Yassaee, Mohammad Mahdi Mojahedian

TL;DR

The paper develops a unified framework of $(G,f)$-divergences and $(G,f)$-information by applying monotone transforms to $f$-divergences, enabling subadditivity analysis for finite alphabets. A central result is the equivalence between divergence subadditivity and information subadditivity under mild smoothness assumptions, facilitated by binary alphabet reductions for broad classes of transforms $G$. Specializing to $G\in\{x,\log(1+x),-\log(1-x)\}$ yields tractable sufficient conditions on $f$ and connects to many standard divergences, including KL, Hellinger, and Jensen–Shannon, with explicit constructions. The framework yields practical operational bounds: finite-blocklength converses, single-letter bounds for binary hypothesis testing, and a generalized sphere-packing exponent for subadditive divergences, broadening the toolbox for non-asymptotic and error-exponent analyses in information theory.

Abstract

We introduce a two-parameter family of discrepancy measures, termed \emph{$(G,f)$-divergences}, obtained by applying a non-decreasing function $G$ to an $f$-divergence $D_f$. Building on Csiszár's formulation of mutual $f$-information, we define a corresponding $(G,f)$-information measure $ I_{G,f}(X;Y)$. A central theme of the paper is subadditivity over product distributions and product channels. We develop reduction principles showing that, for broad classes of $G$, it suffices to verify divergence subadditivity on binary alphabets. Specializing to the functions $G(x)\in\{x,\log(1+x),-\log(1-x)\}$, we derive tractable sufficient conditions on $f$ that guarantee subadditivity, covering many standard $f$-divergences. Finally, we present applications to finite-blocklength converses for channel coding, bounds in binary hypothesis testing, and an extension of the Shannon--Gallager--Berlekamp sphere-packing exponent framework to subadditive $(G,f)$-divergences.

A Class of Subadditive Information Measures and their Applications

TL;DR

The paper develops a unified framework of -divergences and -information by applying monotone transforms to -divergences, enabling subadditivity analysis for finite alphabets. A central result is the equivalence between divergence subadditivity and information subadditivity under mild smoothness assumptions, facilitated by binary alphabet reductions for broad classes of transforms . Specializing to yields tractable sufficient conditions on and connects to many standard divergences, including KL, Hellinger, and Jensen–Shannon, with explicit constructions. The framework yields practical operational bounds: finite-blocklength converses, single-letter bounds for binary hypothesis testing, and a generalized sphere-packing exponent for subadditive divergences, broadening the toolbox for non-asymptotic and error-exponent analyses in information theory.

Abstract

We introduce a two-parameter family of discrepancy measures, termed \emph{-divergences}, obtained by applying a non-decreasing function to an -divergence . Building on Csiszár's formulation of mutual -information, we define a corresponding -information measure . A central theme of the paper is subadditivity over product distributions and product channels. We develop reduction principles showing that, for broad classes of , it suffices to verify divergence subadditivity on binary alphabets. Specializing to the functions , we derive tractable sufficient conditions on that guarantee subadditivity, covering many standard -divergences. Finally, we present applications to finite-blocklength converses for channel coding, bounds in binary hypothesis testing, and an extension of the Shannon--Gallager--Berlekamp sphere-packing exponent framework to subadditive -divergences.
Paper Structure (36 sections, 42 theorems, 380 equations, 2 tables)

This paper contains 36 sections, 42 theorems, 380 equations, 2 tables.

Table of Contents

  1. Introduction
  2. $(G,f)$-divergence and information measure
  3. $(G,f)$-information measure
  4. Data processing and Fano's inequality
  5. Subadditivity of $D_{G,f}$ and $I_{G,f}$
  6. Subadditive functions
  7. A subset of $\mathcal{F}_{\{x\}}$
  8. A subset of $\mathcal{F}_{\{\log(1+x)\}}$
  9. A subset of $\mathcal{F}_{\{-\log(1-x)\}}$
  10. Applications
  11. Error probability in the finite blocklength regime
  12. Binary Hypothesis Testing
  13. Error exponent for the lower bound on the error probability of memoryless channels
  14. Proof of Lemmas \ref{['domain:lem']}--\ref{['lemmaFano']}
  15. Proof of Lemma \ref{['domain:lem']}
  16. ...and 21 more sections

Key Result

Lemma 1

For any distributions $p_X$ and $q_X$ on any set $\mathcal{X}$, we have

Theorems & Definitions (126)

  • Definition 1
  • Definition 2
  • Lemma 1
  • proof
  • Definition 3
  • Definition 4
  • Definition 5
  • Remark 1
  • Example 1
  • Lemma 2
  • ...and 116 more