Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Diagrammatics of free energies with fixed variance for high-dimensional data

Tobias Kühn

TL;DR

The paper builds a non-Gaussian, diagrammatic perturbation theory for free energies at fixed means $m_i$ and variances $v_i$, extending Feynman-diagram methods beyond Gaussian baselines. By introducing and interrelating the Legendre transforms $\Gamma^{K}$ and $\Gamma^{c}$, it derives systematic expansions, cancellations, and resummations for ring and cactus diagrams, enabling complete proofs in rotationally invariant spin models and practical entropy estimates from limited data. The framework unifies forward and inverse problem approaches, clarifies Ising-model perturbative structure, and suggests targeted applications to matrix factorization and higher-order correlations, while outlining avenues for convergence proofs and further theoretical development.

Abstract

Systems with many interacting stochastic constituents are fully characterized by their free energy. Computing this quantity is therefore the objective of various approaches, notably perturbative expansions, which are applied in problems ranging from high-dimensional statistics to complex systems. However, a lot of these techniques are complicated to apply in practice because they lack a sufficient organization of the terms of the perturbative series. In this manuscript, we tackle this problem by using Feynman diagrams, extending a framework introduced earlier to the case of free energies at fixed variances. This diagrammatics do not require the theory to expand around to be Gaussian, which allows its application to the free energy of a spin system studied to derive message-passing algorithms by Maillard et al. 2019. We complete their perturbative derivation of the free energy in the thermodynamic limit. Furthermore, we derive resummations to estimate the entropies of poorly sampled systems requiring only limited statistics and we revisit earlier approaches to compute the free energy of the Ising model, revealing new insights due to the extension of our framework to the free energy at fixed variances. We expect our approach to be particularly useful for problems of high-dimensional statistics, like matrix factorization, and the study of complex networks.

Diagrammatics of free energies with fixed variance for high-dimensional data

TL;DR

The paper builds a non-Gaussian, diagrammatic perturbation theory for free energies at fixed means and variances , extending Feynman-diagram methods beyond Gaussian baselines. By introducing and interrelating the Legendre transforms and , it derives systematic expansions, cancellations, and resummations for ring and cactus diagrams, enabling complete proofs in rotationally invariant spin models and practical entropy estimates from limited data. The framework unifies forward and inverse problem approaches, clarifies Ising-model perturbative structure, and suggests targeted applications to matrix factorization and higher-order correlations, while outlining avenues for convergence proofs and further theoretical development.

Abstract

Systems with many interacting stochastic constituents are fully characterized by their free energy. Computing this quantity is therefore the objective of various approaches, notably perturbative expansions, which are applied in problems ranging from high-dimensional statistics to complex systems. However, a lot of these techniques are complicated to apply in practice because they lack a sufficient organization of the terms of the perturbative series. In this manuscript, we tackle this problem by using Feynman diagrams, extending a framework introduced earlier to the case of free energies at fixed variances. This diagrammatics do not require the theory to expand around to be Gaussian, which allows its application to the free energy of a spin system studied to derive message-passing algorithms by Maillard et al. 2019. We complete their perturbative derivation of the free energy in the thermodynamic limit. Furthermore, we derive resummations to estimate the entropies of poorly sampled systems requiring only limited statistics and we revisit earlier approaches to compute the free energy of the Ising model, revealing new insights due to the extension of our framework to the free energy at fixed variances. We expect our approach to be particularly useful for problems of high-dimensional statistics, like matrix factorization, and the study of complex networks.

Paper Structure

This paper contains 22 sections, 52 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Methods
  3. The Legendre transform with respect to one-point source fields: $\Gamma^{K}$
  4. Derivation of the perturbative expansion for $\Gamma^{K}$
  5. Diagrammatics of $\Gamma^{K}$
  6. Cancellations in fourth order
  7. Cancellations in higher orders
  8. The Gaussian Legendre transform: $\Gamma^{c}$
  9. Derivation of a perturbation expansion for $\Gamma^{c}$
  10. The fourth order of the Gaussian Legendre transform
  11. Ring diagrams in the Gaussian Legendre transform
  12. Applications
  13. Spins coupled by rotationally invariant matrices
  14. The Gaussian Legendre transform as a (maximum) entropy
  15. Ising model
  16. ...and 7 more sections

Figures (3)

  • Figure 1: Sketch of the cancellation mechanism of one-line reducible and cactus diagrams.
  • Figure 2: Sketch of (a) a cactus diagram and (b) an improper pseudo-cactus diagram.
  • Figure 3: Ring diagrams with possible pairs of nodes with identical indices indicated by dashed lines.