A Characterization of Entropy as a Universal Monoidal Natural Transformation

TL;DR

The paper develops a category-theoretic framework in which entropy emerges as a universal monoidal natural transformation between under-categories and codomain monoidal-idempotent-reflections. It shows that Shannon entropy arises as the universal transformation on $\mathsf{LProb}_{\rho}$ (with $0<\rho<1$), while the pairing of Hartley and Shannon entropies arises on $\mathsf{FinProb}$ via universal baseless entropy in $\mathsf{IcOrdCMon}$; these notions extend to a wide array of monoidal categories (sets, vector spaces, abelian groups, augmented simplex, Gaussians) through the universal entropy monad. A key technical achievement is a continuity-free, universal chain rule for conditional entropy, formulated within discretely-indexed MonSiMon categories, which yields the conditional Shannon entropy as a universal functor without requiring topology. The framework unifies disparate entropy concepts under a single reflection-based construction, enabling natural connections between entropies across domains and suggesting new directions for entropy inequalities and potential quantum extensions.

Abstract

We show that the essential properties of entropy (monotonicity, additivity and subadditivity) are consequences of entropy being a monoidal natural transformation from the under category functor $-/\mathsf{LProb}_ρ$ (where $\mathsf{LProb}_ρ$ is category of $ρ$-th-power-summable probability distributions, $0<ρ<1$) to $Δ_{\mathbb{R}}$. Moreover, the Shannon entropy can be characterized as the universal monoidal natural transformation from $-/\mathsf{LProb}_ρ$ to the category of integrally closed partially ordered abelian groups (a reflective subcategory of the lax-slice 2-category over $\mathsf{MonCat}_{\ell}$ in the 2-category of monoidal categories), providing a succinct characterization of Shannon entropy as a reflection arrow. We can likewise define entropy for every monoidal category with a monoidal structure on its under categories (e.g. the category of finite abelian groups, the category of finite inhabited sets, the category of finite dimensional vector spaces, and the augmented simplex category) via the reflection arrow. This implies that all these entropies over different categories are components of a single natural transformation (the unit of the idempotent monad), allowing us to connect these entropies in a natural manner. We also provide a universal characterization of the conditional Shannon entropy based on the chain rule which, unlike the characterization of information loss by Baez, Fritz and Leinster, does not require any continuity assumption.

Figures (2)

• Figure 1.1: The universal entropy of a MonSiMon category $\mathsf{C}\in\mathsf{MonSiMonCat}$ is the MonSiMon functor $\phi_{\mathsf{C}}:\mathsf{C}\to T(\mathsf{C})$ from $\mathsf{C}$ to its reflection $T(\mathsf{C})$ in the reflective subcategory of integrally closed partially ordered (i.c.p.o.) abelian groups, where $T:\mathsf{MonSiMonCat}\to\mathsf{MonSiMonCat}$ is the idempotent monad (the universal entropy monad) sending a category to its reflection, and $\phi:\mathrm{id}_{\mathsf{MonSiMonCat}}\Rightarrow T$ is the monad unit. We will see that the component $\phi_{\mathsf{C}}$ is basically a monoidal natural transformation to a constant functor. We can then use the natural transformation $\phi$ to naturally connect the entropies over various domains, e.g., the (epimorphism wide subcategories of) the opposite category of $\mathsf{FinSet}$ (with entropy given by cardinality), the category of finite-dimensional vector spaces over finite field $\mathbb{F}$ (entropy given by dimension), the category of finite abelian group (entropy given in Section \ref{['subsec:ab']}), the category of finite probability spaces (entropy given by the pairing of the Hartley entropy $H_{0}$ and the Shannon entropy $H_{1}$), the category of finite inhabited sets (entropy given by log cardinality) and the category of $\rho$-th-power-summable discrete probability spaces ($0<\rho<1$, entropy given by Shannon entropy). Refer to Section \ref{['sec:global']} for the descriptions of all the arrows (functors) in the above diagram (which is a commutative diagram in $\mathsf{MonSiMonCat}$).
• Figure 9.1: The universal $\mathsf{V}$-entropies of various categories for various $\mathsf{V}$'s discussed in this paper, placed in a single commutative diagram. The purple objects and morphisms in the purple rectangle are in the category $\mathsf{IcOrdVect}_{\mathbb{Q}}$. The blue and purple objects and morphisms are in $\mathsf{IcOrdAb}$. The green, blue and purple objects and morphisms are in $\mathsf{IcOrdCMon}$. The red, green, blue and purple objects and morphisms are in $\mathsf{OrdCMon}$. All objects and morphisms are in $\mathsf{MonSiMonCat}$. The universal $\mathsf{V}$-entropy of $\mathsf{C}\in\mathsf{MonSiMonCat}$ (where $\mathsf{V}$ is $\mathsf{OrdCMon}$, $\mathsf{IcOrdCMon}$, ...) is the arrow from $\mathsf{C}$ to its reflection in $\mathsf{V}\subseteq\mathsf{MonSiMonCat}$.

Theorems & Definitions (45)

• Definition 1: Monoidal strictly-indexed monoidal categories
• Lemma 2
• proof
• Definition 3: Cancellative ordered commutative monoids kehayopulu1998separativefritz2017resource
• Definition 4: Ordered abelian groups fuchs2011partially
• Definition 5: Ordered vector spaces narici2010topological
• Definition 6: Integrally closed ordered commutative monoids clifford1940partiallyglass1999partially
• Definition 7: $\mathsf{V}$-entropy
• Definition 8: Universal $\mathsf{V}$-entropy
• Proposition 9