Necessary and Sufficient Conditions for Characterizing Finite Discrete Distributions with Generalized Shannon's Entropy
Jialin Zhang
TL;DR
This work resolves when a finite set of Generalized Shannon's Entropy orders $H^{(m)}(p)$ suffices to identify a finite discrete distribution $p$ up to permutation. The authors prove that the mapping $T_M(p)=(H^{(m)}(p))_{m\in M}$ is injective on the sorted probability simplex whenever the number of orders satisfies $r=|M|\ge K-1$, and establish necessary non-injectivity for $r\le K-2$ in the no-multiplicity case; in the binary case a single order suffices. When the distribution has multiplicity $s$ (i.e., $s$ distinct values in $p$), injectivity holds for $r\ge s-1$ and is necessary for $s\ge3$. The proofs combine a $P$-matrix Jacobian analysis, Gale–Nikaidô univalence, and advanced total-positivity tools (STP/SSR/ECT) to build a label-invariant foundation for inference on unordered sample spaces and to motivate practical goodness-of-fit and model-comparison procedures across disparate alphabets.
Abstract
This article establishes necessary and sufficient conditions under which a finite set of Generalized Shannon's Entropy (GSE) characterizes a finite discrete distribution up to permutation. For an alphabet of cardinality K, it is shown that K-1 distinct positive real orders of GSE are sufficient (and necessary if no multiplicity) to identify the distribution up to permutation. When the distribution has a known multiplicity structure with s distinct values, s-1 orders are sufficient and necessary. These results provide a label-invariant foundation for inference on unordered sample spaces and enable practical goodness-of-fit procedures across disparate alphabets. The findings also suggest new approaches for testing, estimation, and model comparison in settings where moment-based and link-based methods are inadequate.