Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Submodularity of Mutual Information for Multivariate Gaussian Sources with Additive Noise

George Crowley, Inaki Esnaola

TL;DR

The paper addresses sensor placement in Gaussian networks by proving that the mutual information $I(X^n; H X^n + Z^k)$ is submodular and non-decreasing under AWGN when $X^n \sim N_n(\mu,\Sigma)$ with $\Sigma \succ 0$. It derives a determinant-based expression for $f(H)$ and proves the three defining properties using block determinant and Schur complement techniques. The main contributions are the established submodularity, the non-decreasing property, and a greedy approximation guarantee of at least $1 - \left( \frac{k-1}{k} \right)^k$ of the optimum (approaching $(e-1)/e$ as $k$ grows). These results enable scalable, near-optimal sensor placement in Gaussian networks with AWGN and Gaussian state distributions.

Abstract

Sensor placement approaches in networks often involve using information-theoretic measures such as entropy and mutual information. We prove that mutual information abides by submodularity and is non-decreasing when considering the mutual information between the states of the network and a subset of $k$ nodes subjected to additive white Gaussian noise. We prove this under the assumption that the states follow a non-degenerate multivariate Gaussian distribution.

Submodularity of Mutual Information for Multivariate Gaussian Sources with Additive Noise

TL;DR

The paper addresses sensor placement in Gaussian networks by proving that the mutual information is submodular and non-decreasing under AWGN when with . It derives a determinant-based expression for and proves the three defining properties using block determinant and Schur complement techniques. The main contributions are the established submodularity, the non-decreasing property, and a greedy approximation guarantee of at least of the optimum (approaching as grows). These results enable scalable, near-optimal sensor placement in Gaussian networks with AWGN and Gaussian state distributions.

Abstract

Sensor placement approaches in networks often involve using information-theoretic measures such as entropy and mutual information. We prove that mutual information abides by submodularity and is non-decreasing when considering the mutual information between the states of the network and a subset of nodes subjected to additive white Gaussian noise. We prove this under the assumption that the states follow a non-degenerate multivariate Gaussian distribution.
Paper Structure (3 sections, 9 theorems, 51 equations)

This paper contains 3 sections, 9 theorems, 51 equations.

Table of Contents

  1. Introduction
  2. Submodularity
  3. Proof of Submodularity

Key Result

Theorem 1

Under the assumption $X^n \sim N_{n}(\hbox{\boldmath$\mu$}, \hbox{\boldmath$\Sigma$})$, where $\hbox{\boldmath$\mu$} \in \mathbb{R}^n$ and $\hbox{\boldmath$\Sigma$} \in S_{++}^{n}$, the function $f\left({\bf H}\right)$ satisfies the following properties:

Theorems & Definitions (14)

  • Definition 1
  • Theorem 1
  • Definition 2: Definition 2.1 nemhauser_analysis_1978
  • Proposition 1: Proposition 2.1 nemhauser_analysis_1978
  • Proposition 2: Proposition 2.2 nemhauser_analysis_1978
  • proof : Proof of condition (1)
  • Lemma 1: Block matrix determinant property
  • Lemma 2: Block matrix inversion
  • Lemma 3
  • Lemma 4: Properties of symmetric positive definite matrices
  • ...and 4 more