On the uniqueness of the coupled entropy

TL;DR

The paper proves that the coupled entropy is the unique information measure that preserves a scale–density correspondence for scale–shape distributions, isolating nonlinear uncertainty through a generalized logarithm of the partition function. It introduces the coupled exponential family and escort moments to define maximizing distributions (coupled stretched exponentials) and demonstrates how this framework yields a scale $\sigma$ that acts as a generalized temperature in coupled thermostatistics. Through composability and extensivity lemmas, the work connects the theory to power-law, stretched exponential, and mixed growth regimes, and contrasts the coupled entropy with Rényi and Tsallis entropies, highlighting its unique alignment with scale-aware uncertainty. The results have implications for nonextensive thermodynamics, statistical learning (including variational inference with coupled free energy), and modeling of complex systems and AI that exhibit heavy tails and nonlinear uncertainty.

Abstract

The coupled entropy is proven to uniquely satisfy the requirement that a generalized entropy be equivalent to the density at the scale for scale-shape distributions. Further, its maximizing distributions, the coupled stretched exponential distributions, are proven to quantify the linear uncertainty with the scale and the nonlinear uncertainty with the shape for a broad class of complex systems. Distributions of the coupled exponentials include the Pareto Types I-IV and Gosset's Student-t. For the Pareto Type II distribution, the Boltzmann-Gibbs-Shannon entropy has a linear dependence on the shape, which dominates over the logarithmic dependence on the scale, motivating the need for a generalization. The Rényi and Tsallis entropies are shown to be of historic importance but ultimately unsatisfactory generalizations. The coupled entropy of the coupled stretched exponential distribution isolates the nonlinear-shape dependence to a generalized logarithm of the partition function. The Rényi and Tsallis entropies retain a strong dependence on the nonlinear-shape such that they are not equivalent to the uncertainty at the scale. Lemmas for the composability and extensivity of the coupled entropy are proven in support of an axiomatic definition. The scope of the coupled entropy includes systems in which the growth of states is power-law, stretched exponential, or a combination.

Figures (6)

• Figure 1: The negative derivative of the score function (logarithm of the distribution) shows the uniqueness of the information scale, $\sigma.$ a) The inverse-scale $(\beta)$ of the $q-$exponential does not have a common intersection. b) The information scale $(\sigma),$ when normalized, has a common intersection independent of the shape $\kappa.$ The $-\sigma \ \mathrm{d}\ln(f(x))/\mathrm{d}x$ function is not dependent on $\sigma.$
• Figure 2: Multiplicative Process Samples 10 samples from the multiplicative noise process defined by equation \ref{['equ_multproc']}, showing the positive values on a logarithmic scale. The NESS distribution is a coupled Gaussian with $\sigma=5$ and a set of couplings $\kappa=(0.1,1,10)$. As the coupling increases, the fluctuations of the process increase, while the scale $\sigma=5$ is independent of the fluctuations. In contrast, the $q$-Gaussian scale $\sqrt{\beta^{-1}}$ is dependent on the multiplicative noise, which undermines its ability to be a measure of generalized temperature.
• Figure 3: Entropies of the coupled exponential distribution are mapped to points on the density. Coupling values of 0.5, 1, and 2 are displayed. The required solution satisfied by the coupled entropy aligns along the scale, a) $\sigma=2,$ and b) $\sigma=0.5.$ BGS with higher entropy corresponds lower density values and a higher value of $x.$ Rényi lower the entropy measure for heavy-tailed distributions via use of the generalized mean. Tsallis further improved the measure via use of a generalized logarithm. Although the normalized Tsallis is structurally closer to the correct solution its high measure of entropy is unsuitable.
• Figure 4: Entropies of the Coupled Exponential Distribution The entropy versus the coupling $(\kappa)$ is shown for the Shannon (gray, dashed), Rényi (black, dashed), Tsallis (blue), and Normalized Tsallis (red) entropies. Scale $(\sigma)$ values of 0.5, 1, and 2 are shown. Shannon is linear and Renyi is logarithmic with the coupling. Both are logarithmic with the scale. Neither the Tsallis or Normalized Tsallis entropies provide a consistent metric of the scale of the distribution. Tsallis has an inverse relationship with the scale, converging to 1, and the Normalized Tsallis multiplies the scale term with the coupling.
• Figure 5: The Coupled Entropy of the double-sided Coupled Stretched Exponential Distribution as a function of the coupling $(\kappa)$. a) With $\gamma=1$ the coupled entropy converges to $Z^\alpha$ as $\kappa \rightarrow \infty.$ b) With $\gamma=\alpha$ the coupled entropy converges to $Z$ as $\kappa \rightarrow \infty.$ Each graph shows three values of the variable power, $\alpha$, 0.5 - orange, 1.0 - blue, 1.5 - magenta, and three values of the scale, $\sigma$, 0.25 - dashed, 0.5 - line, 1.0 - dash-dotted. For $\alpha=1$ (blue), a coupled exponential, $Z=\sigma$ and the coupled entropy is only slightly dependent on the coupling. For $\alpha=2$ (magenta), a coupled Gaussian, the normalization has a strong dependence on the coupling, which is amplified by the power 2; thus, taking the root $\gamma=\alpha$ may form a better metric. However, this same root amplifies the metric for $\alpha=\frac{1}{2}$ and low values of $\kappa$.

Theorems & Definitions (26)

• Definition 1: Informational Scale
• Example 1: Informational scale of common heavy-tailed distributions
• Definition 2: Coupled Exponential Distribution
• Definition 3: Coupled Gaussian Distribution
• Lemma 1: Independent Properties of the CESD Parameters
• proof
• Definition 4: Required Generalized Entropy Solution
• Lemma 2: Independent Equal Moments
• proof
• Definition 5: The Coupled Entropy