Relative Entropy and Mutual Information in Gaussian Statistical Field Theory

TL;DR

It is demonstrated that the relative entropy depends crucially on $d$ the dimension of Euclidean space, and it is argued that the properties of mutual information in scalar field theories can be explained by the Markov property of these theories.

Abstract

Relative entropy is a powerful measure of the dissimilarity between two statistical field theories in the continuum. In this work, we study the relative entropy between Gaussian scalar field theories in a finite volume with different masses and boundary conditions. We show that the relative entropy depends crucially on $d$, the dimension of Euclidean space. Furthermore, we demonstrate that the mutual information between two disjoint regions in $\mathbb{R}^d$ is finite if the two regions are separated by a finite distance and satisfies an area law. We then construct an example of "touching" regions between which the mutual information is infinite. We argue that the properties of mutual information in scalar field theories can be explained by the Markov property of these theories.

Figures (9)

• Figure 1: The relative entropy between two field theories with masses $m_1$ and $m_2$, respectively, on an interval of length $L$. We consider Dirichlet and Neumann boundary conditions and plot the relative entropy in bits against the masses in units of the inverse interval length $L^{-1}$. Note that in the limit $m_i \to 0$ the relative entropy is finite for Dirichlet boundary conditions but diverges for Neumann boundary conditions. This is due to the zero mode of the Laplacian that is present when we choose Neumann boundary conditions.
• Figure 2: The relative entropy between two field theories with masses $m_1$ and $m_2 = 30 L^{-1}$, respectively, on an interval of length $L$. We consider Dirichlet, periodic and Neumann boundary conditions and plot the relative entropy in units of bits against the inverse mass (or correlation length) $m_1^{-1}$ in units of the interval length $L$. We observe the ordering $D^\mathrm{D}_\mathrm{KL} \leq D^\mathrm{P}_\mathrm{KL} \leq D^\mathrm{N}_\mathrm{KL}$, with equality only for coinciding masses.
• Figure 3: (a) The relative entropy between two field theories with masses $m_1$ and $m_2 = \frac{1}{2} m_1$, respectively, on an interval of length $L$ for three different boundary conditions. The relative entropy in units of bits is plotted against the system size $L$ in units of the correlation length or inverse mass $m_1^{-1}$. We see that as soon as the system size is larger than the largest correlation length (in this case, $m_2^{-1} = 2 m_1^{-1}$), the relative entropy scales linearly with $L$. If $L$ is smaller than both correlation lengths, the relative entropy is close to zero for Dirichlet boundary conditions and attains a constant value for Neumann and periodic boundary conditions. (b) The relative entropy density as a function of the system size $L$. We see that in the infinite volume limit $L \to + \infty$, the relative entropy density converges for all three boundary conditions to the limit given in \ref{['eq:infinite_vol_density']}, represented in the Figure as a grey dash-dotted line.
• Figure 4: (a) The relative entropy between two field theories with masses $m_1$ and $m_2 = 30 L^{-1}$ on a $d$-cube of edge length $L$ in dimensions $d=1$ (blue) and $d=2$ (red). We plot the relative entropy in units of bits against the correlation length or inverse mass $m_1^{-1}$ in units of the edge length $L$. The lower (upper) bound of each shaded region represents the Dirichlet (Neumann) relative entropy. We observe that for a fixed edge length $L$ the relative entropy is larger in higher dimensions. (b) Logarithmic plot of the relative entropy to include the case $d=3$ (orange). We use the same parameters as in (a).
• Figure 5: The relative entropy between a one-dimensional field theory with free boundary conditions and a field theory with $\sigma$-boundary conditions given in \ref{['eq:sigma_bcs']}, both with mass $m$, on an interval $\Omega = (0,1)$. As indicated by the arrows, the special values $\sigma = 0$, $\sigma = m$ and $\sigma = + \infty$ correspond to Neumann, free and Dirichlet boundary conditions, respectively. The relative entropy scales linearly in $\sigma$ for large $\sigma$, indicating that a $\sigma$-boundary condition field is mutually singular to a Dirichlet field.

Theorems & Definitions (65)

• Theorem : Theorem \ref{['thm:main_1']}
• Theorem : Theorems \ref{['thm:main_2']} and \ref{['thm:main_3']}
• Theorem : Theorems \ref{['thm:equiv_mutual_info']} and \ref{['thm:main_5']}
• Lemma 1: Guerra1976, see also Glimm2012
• Definition 1: Bogachev2007
• Definition 2: Kullback1951
• Lemma 2
• proof
• Theorem 3
• Theorem 4