Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Description Complexity of Unary Structures in First-Order Logic with Links to Entropy

Reijo Jaakkola, Antti Kuusisto, Miikka Vilander

TL;DR

This work defines structures with FO-formulas that are strictly linear in the size of the model as opposed to using the naive quadratic ones, and uses arguments based on formula size games to obtain related lower bounds for description complexity.

Abstract

The description complexity of a model is the length of the shortest formula that defines the model. We study the description complexity of unary structures in first-order logic FO, also drawing links to semantic complexity in the form of entropy. The class of unary structures provides, e.g., a simple way to represent tabular Boolean data sets as relational structures. We define structures with FO-formulas that are strictly linear in the size of the model as opposed to using the naive quadratic ones, and we use arguments based on formula size games to obtain related lower bounds for description complexity. For a typical structure the upper and lower bounds in fact match up to a sublinear term, leading to a precise asymptotic result on the expected description complexity of a randomly selected structure. We then give bounds on the relationship between Shannon entropy and description complexity. We extend this relationship also to Boltzmann entropy by establishing an asymptotic match between the two entropies. Despite the simplicity of unary structures, our arguments require the use of formula size games, Stirling's approximation and Chernoff bounds.

Description Complexity of Unary Structures in First-Order Logic with Links to Entropy

TL;DR

This work defines structures with FO-formulas that are strictly linear in the size of the model as opposed to using the naive quadratic ones, and uses arguments based on formula size games to obtain related lower bounds for description complexity.

Abstract

The description complexity of a model is the length of the shortest formula that defines the model. We study the description complexity of unary structures in first-order logic FO, also drawing links to semantic complexity in the form of entropy. The class of unary structures provides, e.g., a simple way to represent tabular Boolean data sets as relational structures. We define structures with FO-formulas that are strictly linear in the size of the model as opposed to using the naive quadratic ones, and we use arguments based on formula size games to obtain related lower bounds for description complexity. For a typical structure the upper and lower bounds in fact match up to a sublinear term, leading to a precise asymptotic result on the expected description complexity of a randomly selected structure. We then give bounds on the relationship between Shannon entropy and description complexity. We extend this relationship also to Boltzmann entropy by establishing an asymptotic match between the two entropies. Despite the simplicity of unary structures, our arguments require the use of formula size games, Stirling's approximation and Chernoff bounds.
Paper Structure (9 sections, 18 theorems, 29 equations, 1 figure)

This paper contains 9 sections, 18 theorems, 29 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related work, techniques and applications
  4. Preliminaries
  5. Upper bound formulas
  6. Lower bounds via formula size games
  7. Expected description complexity
  8. Entropy and description complexity
  9. Conclusion

Key Result

Theorem 1

Let $\mathfrak{M}$ be a model of size $n$. Let $T = \{\pi_1, \dots, \pi_\ell\}$ be the types realized in $\mathfrak{M}$, enumerated in ascending order of numbers of realizing points. Now we have

Figures (1)

  • Figure 1: Figure \ref{['fig:entropy']} on the left shows an area that encapsulates all combinations of Shannon entropy and $\mathrm{FO}$-description complexity for the values $|\tau| = 2$ and $n = 1000$. Figure \ref{['fig:d-entropy']} on the right concerns the case of $\mathrm{FO}_d$ and shows bounds on description complexity in terms of Boltzmann entropy for values $|\tau| = 2$, $n = 100$ and $d= 10$ with the constants $-3$ and $c_\tau$ omitted.

Theorems & Definitions (33)

  • Theorem 1
  • proof
  • Corollary 1
  • proof
  • Theorem 2
  • proof
  • Corollary 2
  • Theorem 3
  • proof
  • Lemma 1
  • ...and 23 more