Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tsallis Entropy derived from the Chaitin-Kolmogorov Informational Entropy

Airton Deppman

TL;DR

This work addresses deriving Tsallis entropy from a microscopic basis in algorithmic information theory by imposing grammar constraints on strings produced by a universal Turing machine. Using Chaitin-Kolmogorov principles, the authors show that restricted grammar induces a power-law scaling of the algorithmic cost, leading to a non-additive entropy $S_q$ with $q=1-1/\alpha$. They connect this framework to linguistic laws (Zipf and Heaps), validate it with simulations showing power-law behavior, and extend the implications to a generalized Landauer limit and a generalized incompressible Omega number $Ω_q$, thereby providing a first-principles foundation for Tsallis statistics in constrained computational spaces. The results offer a mechanistic view of how syntactic rules shape information density and complexity, with potential impact on understanding dissipation and undecidability in constrained systems.

Abstract

We provide a rigorous first-principle derivation of the non-additive Tsallis' entropy by employing the Chaitin-Kolmogorov algorithmic information theory. By applying non-local restrictive rules on the string formation (grammar), we show that the algorithmic cost follows a power-law of the string length, instead of the linear behaviour obtained in the classical theory. As a result, the Tsallis entropy governs the increase of information. We explore the result showing, through Landauer's limit, that the heat dissipation in systems with long-range correlations is diminished. The $Ω_q$ number, which remains incompressible, now offers the possibility of a continuous increase of complexity, measured by the parameter $q$. We show the consistency of the results by a numerical simulation, and discuss Zipf's law in light of the new findings.

Tsallis Entropy derived from the Chaitin-Kolmogorov Informational Entropy

TL;DR

This work addresses deriving Tsallis entropy from a microscopic basis in algorithmic information theory by imposing grammar constraints on strings produced by a universal Turing machine. Using Chaitin-Kolmogorov principles, the authors show that restricted grammar induces a power-law scaling of the algorithmic cost, leading to a non-additive entropy with . They connect this framework to linguistic laws (Zipf and Heaps), validate it with simulations showing power-law behavior, and extend the implications to a generalized Landauer limit and a generalized incompressible Omega number , thereby providing a first-principles foundation for Tsallis statistics in constrained computational spaces. The results offer a mechanistic view of how syntactic rules shape information density and complexity, with potential impact on understanding dissipation and undecidability in constrained systems.

Abstract

We provide a rigorous first-principle derivation of the non-additive Tsallis' entropy by employing the Chaitin-Kolmogorov algorithmic information theory. By applying non-local restrictive rules on the string formation (grammar), we show that the algorithmic cost follows a power-law of the string length, instead of the linear behaviour obtained in the classical theory. As a result, the Tsallis entropy governs the increase of information. We explore the result showing, through Landauer's limit, that the heat dissipation in systems with long-range correlations is diminished. The number, which remains incompressible, now offers the possibility of a continuous increase of complexity, measured by the parameter . We show the consistency of the results by a numerical simulation, and discuss Zipf's law in light of the new findings.
Paper Structure (8 sections, 36 equations, 1 figure, 1 table)

This paper contains 8 sections, 36 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. The algorithmic approach to entropy
  3. Introducing complexity
  4. Linguistic foundations of the power law distribution
  5. Simulation of grammar effects
  6. Applications
  7. Conclusions
  8. Acknowledgements

Figures (1)

  • Figure 1: Results of the number of allowed strings of length $L$ under four different rules: 1) completely aleatory (blue solid line); 2) local Markovian sequence (dashed orange line); 3) recursive nesting, introducing non-local correlations; 4) global structure, which introduces long-range correlations.