Coupled Entropy: A Goldilocks Generalization for Complex Systems

TL;DR

The paper identifies critical conceptual flaws in Tsallis $q$-statistics and proposes the coupled entropy, grounded in the generalized Pareto distribution and the coupled exponential family, to separate nonlinear coupling (shape) from linear uncertainty (scale) in complex systems. It develops the coupled entropy, cross-entropy, and divergences, proves a maximum-entropy principle yielding the coupled exponential as the maximizing distribution, and links these ideas to a generalized Boltzmann–Gibbs framework with a scale-based generalized temperature. Through comparisons with Tsallis and normalized Tsallis entropies, the work shows that the coupled entropy uniquely balances scale while accounting for nonlinearity, with limits that recover the scale $σ$ as coupling grows. The paper also discusses practical implications for robust variational inference and learning, illustrating how independent-equals sampling can enable heavy-tailed or compact-support models and proposing a coupled free-energy objective to improve stability in training generative models.

Abstract

The coupled entropy is proven to correct a flaw in the derivation of the Tsallis entropy and thereby solidify the theoretical foundations for analyzing the uncertainty of complex systems. The Tsallis entropy originated from considering power probabilities $p_i^q$ in which \textit{q} independent, identically-distributed random variables share the same state. The maximum entropy distribution was derived to be a \textit{q}-exponential, which is a member of the shape ($κ$), scale ($σ$) distributions. Unfortunately, the $q$-exponential parameters were treated as though valid substitutes for the shape and scale. This flaw causes a misinterpretation of the generalized temperature and an imprecise derivation of the generalized entropy. The coupled entropy is derived from the generalized Pareto distribution (GPD) and the Student's t distribution, whose shape derives from nonlinear sources and scale derives from linear sources of uncertainty. The Tsallis entropy of the GPD converges to one as $κ\rightarrow\infty$, which makes it too cold. The normalized Tsallis entropy (NTE) introduces a nonlinear term multiplying the scale and the coupling, making it too hot. The coupled entropy provides perfect balance, ranging from $\ln σ$ for $κ=0$ to $σ$ as $κ\rightarrow\infty$. One could say, the coupled entropy allows scientists, engineers, and analysts to eat their porridge, confident that its measure of uncertainty reflects the mathematical physics of the scale of non-exponential distributions while minimizing the dependence on the shape or nonlinear coupling. Examples of complex systems design including a coupled variation inference algorithm are reviewed.

Figures (2)

• Figure 1: The q-exponential distribution (left) does not share a common intersection as the scale is changed. For the coupled exponential distribution (GPD) (right) normalization by the scale assures that the full family instersects at (1,1).
• Figure 2: Comparison of generalized entropies of the coupled exponential (generalized Pareto) distribution. The Tsallis entropy (blue) is too cold, converging to one as the coupling increases. The normalized Tsallis entropy (red) is too hot, increasing to infinity. Only the coupled entropy converges to the scale as the coupling increases.

Theorems & Definitions (11)

• Definition 1: Generalized Pareto Distribution
• Definition 2: Coupled Survival Function
• Proposition 1: Coupled Density Functions
• Lemma 1: Multiplicative Process with Coupled Gaussian Limit
• Definition 3: Coupled Entropy
• Definition 4: Coupled Divergence
• Lemma 2: Coupled Entropy Maximized by the Coupled Exponential Distribution
• proof
• Definition 5: Coupled Boltzmann-Gibbs distribution
• proof : Lemma 1 Proof: Multiplicative Process with Coupled Gaussian Limit