Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A complete characterisation of conditional entropies

Roberto Rubboli, Erkka Haapasalo, Marco Tomamichel

TL;DR

This work provides a complete axiomatic characterization of conditional entropies by proving that any operationally meaningful conditional entropy can be expressed as a barycenter of extremal quantities $H_{t,\tau}$ and their limits, i.e., exponential averages of conditioned Rényi entropies. The authors build a semiring framework for conditional majorization to analyze large-copy transformations and catalytic processes, linking transformation rates to these entropies. They establish sufficient and, under regularity assumptions, necessary conditions on the parameter measures to ensure monotonicity under conditionally mixing channels, and they derive a rate formula and second laws for thermodynamics with side information. The results unify and subsume many conditional entropies in the literature as special cases of the $H_{t,\tau}$ family, and they clarify when these quantities produce valid conditional entropies with operational meaning in multi-copy transformations and resource theories.

Abstract

Entropies are fundamental measures of uncertainty with central importance in information theory and statistics and applications across all the quantitative sciences. Under a natural set of operational axioms, the most general form of entropy is captured by the family of Rényi entropies, parameterized by a real number $α$. Conditional entropy extends the notion of entropy by quantifying uncertainty from the viewpoint of an observer with access to potentially correlated side information. However, despite their significance and the emergence of various useful definitions, a complete characterization of measures of conditional entropy that satisfy a natural set of operational axioms has remained elusive. In this work, we provide a complete characterization of conditional entropy, defined through a set of axioms that are essential for any operationally meaningful definition: additivity for independent random variables, invariance under relabeling, and monotonicity under conditional mixing channels. We prove that the most general form of conditional entropy is captured by a family of measures that are exponential averages of Rényi entropies of the conditioned distribution and parameterized by a real parameter and a probability measure on the positive reals. Finally, we show that these quantities determine the rate of transformation under conditional mixing and provide a set of second laws of quantum thermodynamics with side information for states diagonal in the energy eigenbasis.

A complete characterisation of conditional entropies

TL;DR

This work provides a complete axiomatic characterization of conditional entropies by proving that any operationally meaningful conditional entropy can be expressed as a barycenter of extremal quantities and their limits, i.e., exponential averages of conditioned Rényi entropies. The authors build a semiring framework for conditional majorization to analyze large-copy transformations and catalytic processes, linking transformation rates to these entropies. They establish sufficient and, under regularity assumptions, necessary conditions on the parameter measures to ensure monotonicity under conditionally mixing channels, and they derive a rate formula and second laws for thermodynamics with side information. The results unify and subsume many conditional entropies in the literature as special cases of the family, and they clarify when these quantities produce valid conditional entropies with operational meaning in multi-copy transformations and resource theories.

Abstract

Entropies are fundamental measures of uncertainty with central importance in information theory and statistics and applications across all the quantitative sciences. Under a natural set of operational axioms, the most general form of entropy is captured by the family of Rényi entropies, parameterized by a real number . Conditional entropy extends the notion of entropy by quantifying uncertainty from the viewpoint of an observer with access to potentially correlated side information. However, despite their significance and the emergence of various useful definitions, a complete characterization of measures of conditional entropy that satisfy a natural set of operational axioms has remained elusive. In this work, we provide a complete characterization of conditional entropy, defined through a set of axioms that are essential for any operationally meaningful definition: additivity for independent random variables, invariance under relabeling, and monotonicity under conditional mixing channels. We prove that the most general form of conditional entropy is captured by a family of measures that are exponential averages of Rényi entropies of the conditioned distribution and parameterized by a real parameter and a probability measure on the positive reals. Finally, we show that these quantities determine the rate of transformation under conditional mixing and provide a set of second laws of quantum thermodynamics with side information for states diagonal in the energy eigenbasis.
Paper Structure (32 sections, 33 theorems, 274 equations, 1 figure)

This paper contains 32 sections, 33 theorems, 274 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Definitions and preliminary results
  3. Definitions relevant for entropies
  4. Definitions relevant for conditional entropies
  5. The set of entropies
  6. The semiring of conditional majorization
  7. Background on preordered semirings and a Vergleichsstellensatz
  8. Definition of the conditional majorization semiring
  9. Invariance of the homomorphisms under embedding and shifting
  10. Some important properties of the semiring $S$
  11. The homomorphisms
  12. The derivations
  13. Sufficient conditions for the parameters
  14. Sufficient conditions for the discrete case
  15. Sufficient conditions for the temperate homomorphisms - general measure
  16. ...and 17 more sections

Key Result

Lemma 2.5

Let $P_X$ and $Q_X$ be two $d$-dimensional vectors with nonnegative entries. Moreover, let $P_X^{\downarrow}$ and $Q_X^{\downarrow}$ denote the vectors obtained by rearranging the components of $P_X$ and $Q_X$, respectively, in non-increasing order. Then $P_X \succeq Q_X$ if and only if and equality holds for $k = d$, that is, $\sum_{i=1}^{d} P_X^{\downarrow}(i) = \sum_{i=1}^{d} Q_X^{\downarrow}(

Figures (1)

  • Figure 1: The figure shows the lines defined in equation \ref{['eq: lines necessary']}, and the dominant contributions in equation \ref{['eq: contributions lines']} in the different ranges.

Theorems & Definitions (79)

  • Definition 2.1: Embedding and relabeling of probability distributions
  • Definition 2.2: Doubly stochastic channel
  • Definition 2.3: Majorization
  • Remark 2.4
  • Lemma 2.5: Initial sum condition
  • Definition 2.6: Entropy
  • Definition 2.7: Conditionally mixing channel
  • Definition 2.8: Embedding and relabeling of joint probability distributions
  • Definition 2.9: Conditional majorization
  • Remark 2.10
  • ...and 69 more