Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Diagrammatics of information

Mee Seong Im, Clement Kam, Caden Pici

TL;DR

This work develops a diagrammatic framework for Shannon entropy by embedding entropy in Cathelineau's infinitesimal dilogarithm $J( extbf{k})$ and its beta-entropy counterpart $eta( extbf{k})$, establishing a concrete isomorphism between them. It then builds a diagrammatic calculus with boundary walls and cobordisms to represent single-variable, joint, and conditional entropy, yielding invariant expressions and a natural encoding of entropy via additive and multiplicative network components. The paper also proves two complete deformations showing that the $5$-term dilogarithm degenerates to the $4$-term infinitesimal dilogarithm, using tangent- and dual-number techniques, linking topological/diagrammatic methods to classical information theory. Collectively, these results unify information-theoretic quantities with algebraic and topological structures, enabling new invariants and potential extensions to generalized entropies and higher-dimensional diagrammatics.

Abstract

We introduce a diagrammatic perspective for Shannon entropy created by the first author and Mikhail Khovanov and connect it to information theory and mutual information. We also give two complete proofs that the $5$-term dilogarithm deforms to the $4$-term infinitesimal dilogarithm.

Diagrammatics of information

TL;DR

This work develops a diagrammatic framework for Shannon entropy by embedding entropy in Cathelineau's infinitesimal dilogarithm and its beta-entropy counterpart , establishing a concrete isomorphism between them. It then builds a diagrammatic calculus with boundary walls and cobordisms to represent single-variable, joint, and conditional entropy, yielding invariant expressions and a natural encoding of entropy via additive and multiplicative network components. The paper also proves two complete deformations showing that the -term dilogarithm degenerates to the -term infinitesimal dilogarithm, using tangent- and dual-number techniques, linking topological/diagrammatic methods to classical information theory. Collectively, these results unify information-theoretic quantities with algebraic and topological structures, enabling new invariants and potential extensions to generalized entropies and higher-dimensional diagrammatics.

Abstract

We introduce a diagrammatic perspective for Shannon entropy created by the first author and Mikhail Khovanov and connect it to information theory and mutual information. We also give two complete proofs that the -term dilogarithm deforms to the -term infinitesimal dilogarithm.

Paper Structure

This paper contains 18 sections, 25 theorems, 97 equations, 19 figures.

Table of Contents

  1. Introduction
  2. Background
  3. Cathelineau's k-vector space
  4. Vector space isomorphic to Shannon's entropy
  5. Isomorphism of Jk and betak
  6. Shannon entropy and and Cathelineau's vector space
  7. Shannon entropy
  8. Joint entropy
  9. Joint entropy of 2 random variables
  10. Entropy for u random variables
  11. Rescaling a finite probability distribution by a scalar
  12. Diagrammatics of Shannon entropy
  13. Boundary wall
  14. Connections from diagrammatics to J(k)
  15. Diagrammatics of conditional entropy
  16. ...and 3 more sections

Key Result

Lemma 2.1

In the vector space $J(\mathbf{k})$, we have $\langle a,0\rangle =\langle 0,a\rangle =0$, and $\langle a,-a\rangle = 0$.

Figures (19)

  • Figure 3.1.1: Entropy function $H(p)=-p\log |p|-(1-p)\log |1-p|$ for $-1\leq p\leq 2$.
  • Figure 5.0.1: Upper left: Whenever two black additive lines merge, we evaluate $\langle a,b\rangle$ at the additive vertex. Upper middle: whenever two additive lines split, we evaluate $-\langle a,b\rangle$. Upper right: we are allowed to have virtual crossing whenever two additive lines cross but the intersection of these two lines is virtual, so there is no corresponding evaluation for these two additive lines. Bottom left: whenever a red line is to the left of a black line, then we rescale the value $a$ of the additive line by $c\in \mathbf{k}^*$ of the multiplicative line. Bottom middle: multiplicative lines can merge, resulting in multiplication of their weights. Bottom right: we may have a red vertex on the multiplicative network, resulting in the swapping of co-orientations of the multiplicative line.
  • Figure 5.0.2: When additive lines are pointing downwards, we rotate the additive vertices in the network whilst fixing the boundary points so that the orientations at the additive vertices are upwards. Top row: the additive vertex in each of the figures gives the contribution of $\langle a,b\rangle$. Bottom row: the additive vertices give $-\langle a,b\rangle$.
  • Figure 5.0.3: Top left: This diagram represents the $2$-cocycle condition \ref{['item:Cath_cocycle']} in Cathelineau's $\mathbf{k}$-vector space. Top right: Two lines crossing is virtual, but the additive vertex contributes $\langle b,a\rangle = \langle a,b \rangle$. Bottom left: the additive vertex on the left hand side gives $\langle ac, bc\rangle$ while the additive vertex on the right hand side gives $c\langle a,b\rangle$. This cobordism implies that they are equal. Bottom right: These two isotopies imply that one may pull the red multiplicative line away from black additive line.
  • Figure 5.0.4: When $\mathbf{k}=\mathbb{R}$ and $p_1+p_2 +p_3 =1$, each diagram on the left and the right evaluates to Shannon entropy.
  • ...and 14 more figures

Theorems & Definitions (46)

  • Lemma 2.1
  • proof
  • Lemma 2.2
  • proof
  • Lemma 2.3
  • proof
  • Lemma 3.1
  • proof
  • Proposition 3.2
  • proof
  • ...and 36 more