Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cumulants of multiinformation density in the case of a multivariate normal distribution

Guillaume Marrelec, Alain Giron

TL;DR

The paper introduces multiinformation density $i_d$ for partitioned Gaussian vectors and derives a closed-form cumulant-generating function in terms of the block regression-coefficient matrix $\boldsymbol{\Gamma}$. It shows $\kappa_1(i_d) = I(\boldsymbol{X}_1; \dots; \boldsymbol{X}_N)$ and $\kappa_\ell(i_d) = \frac{(\ell-1)!}{2} \mathrm{tr}(\boldsymbol{\Gamma}^\ell)$ for $\ell \ge 2$, with an explicit expression for the variance, highlighting that $i_d$ and its moments depend only on correlations (via $\boldsymbol{\Gamma}$) and are invariant to marginal variances. A graphical interpretation connects $\mathrm{tr}(\boldsymbol{\Gamma}^l)$ to directed loops in a fully connected dependence graph, and special cases (notably $N=2$) recover classical results on mutual information, canonical correlations, and correlation-based measures. The findings suggest a natural parameterization of dependence through $\boldsymbol{\Gamma}$ and open avenues for estimators and high-dimensional analysis, while noting limitations for non-Gaussian distributions and potential generalizations to other divergence forms.

Abstract

We consider a generalization of information density to a partitioning into $N \geq 2$ subvectors. We calculate its cumulant-generating function and its cumulants, showing that these quantities are only a function of all the regression coefficients associated with the partitioning.

Cumulants of multiinformation density in the case of a multivariate normal distribution

TL;DR

The paper introduces multiinformation density for partitioned Gaussian vectors and derives a closed-form cumulant-generating function in terms of the block regression-coefficient matrix . It shows and for , with an explicit expression for the variance, highlighting that and its moments depend only on correlations (via ) and are invariant to marginal variances. A graphical interpretation connects to directed loops in a fully connected dependence graph, and special cases (notably ) recover classical results on mutual information, canonical correlations, and correlation-based measures. The findings suggest a natural parameterization of dependence through and open avenues for estimators and high-dimensional analysis, while noting limitations for non-Gaussian distributions and potential generalizations to other divergence forms.

Abstract

We consider a generalization of information density to a partitioning into subvectors. We calculate its cumulant-generating function and its cumulants, showing that these quantities are only a function of all the regression coefficients associated with the partitioning.

Paper Structure

This paper contains 15 sections, 1 theorem, 56 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Proof of theorem
  3. Cumulant-generating function
  4. Cumulants
  5. Consequences
  6. Partitioning into two subvectors
  7. Irrelevance of variances
  8. Graphical interpretation
  9. Discussion
  10. Generalization
  11. Future work
  12. Mutual independence, $i_d$ and multiinformation
  13. Positive-definiteness of $\boldsymbol{\Sigma} ^ { - 1 } - t \boldsymbol{\Phi}$
  14. Alternative expression of multiinformation
  15. Checking asymptotic normality

Key Result

Theorem 1

Let $\boldsymbol{X}$ be a $d$-dimensional variable following a multivariate normal distribution with mean $\boldsymbol{\mu}$ and covariance matrix $\boldsymbol{\Sigma}$. Partition $\boldsymbol{X}$ into $N$ subvectors $( \boldsymbol{X}_1, \dots, \boldsymbol{X}_N )$, and set $i_d$ the corresponding mu where is the block matrix whose diagonal blocks are equal to $\boldsymbol{0}$ and where each off-d

Figures (1)

  • Figure 1: Graphical interpretation of $\mathrm{tr} ( \boldsymbol{\Gamma} ^ l )$. We consider the case $N = 4$ and $l = 3$. The directed 3-loop $lp = (1 \to 2 \to 3 \to 1 )$ is represented with dark arrows. The value of $\tau$ on this loop is equal to $\tau ( l ) = \mathrm{tr} ( \boldsymbol{\Gamma}_{ 1 | 3 } \boldsymbol{\Gamma}_{ 3 | 2 } \boldsymbol{\Gamma}_{ 2 | 1 } )$. $\mathrm{tr} ( \boldsymbol{\Gamma} ^ 3 )$ is obtained by summing $\tau ( p )$ over all 3-loops.

Theorems & Definitions (1)

  • Theorem 1