Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mutual Information Estimation via Score-to-Fisher Bridge for Nonlinear Gaussian Noise Channels

Tadashi Wadayama

TL;DR

This work addresses estimating mutual information in nonlinear Gaussian noise channels without posterior access by establishing a Score-to-Fisher bridge. It uses denoising score matching to recover the score of the channel output, from which Fisher information is estimated and integrated via a Fisher integral representation to obtain $I(X;Y_T)$, exploiting the equivalence $I(X;Y_t)=I(W;Y_t)$ with $W=f(X)$ for deterministic front-ends. The framework includes two practical training schemes (per-$t$ and noise-conditional), a robust log-domain trapezoid integration with tail corrections, and is validated on Gaussian and nonlinear channels against analytic solutions and KDE baselines, showing high accuracy and scalability. The approach avoids posterior computations and partitions functions, offering a scalable, channel-agnostic tool with direct relevance to coding, preprocessing, and sensing system design where nonlinear front-ends are present.

Abstract

We present a numerical method to evaluate mutual information (MI) in nonlinear Gaussian noise channels by using denoising score matching (DSM) learning for estimating the score function of channel output. Via de Bruijn's identity, Fisher information estimated from the learned score function yields accurate estimates of MI through a Fisher integral representation for a variety of priors and channel nonlinearities. In this work, we propose a comprehensive theoretical foundation for the Score-to-Fisher bridge methodology, along with practical guidelines for its implementation. We also conduct extensive validation experiments, comparing our approach with closed-form solutions and a kernel density estimation baseline. The results of our numerical experiments demonstrate that the proposed method is both practical and efficient for MI estimation in nonlinear Gaussian noise channels.

Mutual Information Estimation via Score-to-Fisher Bridge for Nonlinear Gaussian Noise Channels

TL;DR

This work addresses estimating mutual information in nonlinear Gaussian noise channels without posterior access by establishing a Score-to-Fisher bridge. It uses denoising score matching to recover the score of the channel output, from which Fisher information is estimated and integrated via a Fisher integral representation to obtain , exploiting the equivalence with for deterministic front-ends. The framework includes two practical training schemes (per- and noise-conditional), a robust log-domain trapezoid integration with tail corrections, and is validated on Gaussian and nonlinear channels against analytic solutions and KDE baselines, showing high accuracy and scalability. The approach avoids posterior computations and partitions functions, offering a scalable, channel-agnostic tool with direct relevance to coding, preprocessing, and sensing system design where nonlinear front-ends are present.

Abstract

We present a numerical method to evaluate mutual information (MI) in nonlinear Gaussian noise channels by using denoising score matching (DSM) learning for estimating the score function of channel output. Via de Bruijn's identity, Fisher information estimated from the learned score function yields accurate estimates of MI through a Fisher integral representation for a variety of priors and channel nonlinearities. In this work, we propose a comprehensive theoretical foundation for the Score-to-Fisher bridge methodology, along with practical guidelines for its implementation. We also conduct extensive validation experiments, comparing our approach with closed-form solutions and a kernel density estimation baseline. The results of our numerical experiments demonstrate that the proposed method is both practical and efficient for MI estimation in nonlinear Gaussian noise channels.

Paper Structure

This paper contains 34 sections, 4 theorems, 43 equations, 6 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Notation
  4. Related works
  5. AWGN channel and score function
  6. I--MMSE relation
  7. de Bruijn identity
  8. Denoising score matching (DSM) method
  9. Estimating Mutual Information in AWGN Channels
  10. MI--Fisher relation
  11. Fisher Integral representations of mutual information
  12. Estimation of Fisher Information
  13. Gaussian input case for validation
  14. Linear Gaussian channel case
  15. Estimation of Mutual Information
  16. ...and 19 more sections

Key Result

Proposition 1

For all $t>0$, the identity holds guo2005palomar2006reeves2018.

Figures (6)

  • Figure 1: Fisher information $J(Y_t)$ versus noise variance $t$ for Gaussian input with $P=1$ and dimensions $n \in \{4, 8, 16\}$. Solid lines with circular markers show ground truth values $J(Y_t) = n/(P+t)$, while dashed lines with triangular markers show estimated values $\hat{J}(Y_t)$.
  • Figure 2: Mutual information (per symbol) $I(X;Y_t)/n$ versus noise variance $t$ for Gaussian input.
  • Figure 3: Noise-conditional model for mutual information estimation results for Gaussian input ($n=4$, $P=1$). Scheme B was used.
  • Figure 4: Mutual information estimation for BPSK input ($n=1$, $P=1$). Dotted line: Gaussian input theoretical baseline; solid circles: BPSK exact MI via numerical integration; dashed squares: DSM estimates.
  • Figure 5: Mutual information estimation for linear Gaussian channel $Y_t = {\bm A} X + Z_t$ ($n=4$, $P=1$). Circles: analytical ground truth; triangles: DSM estimates.
  • ...and 1 more figures

Theorems & Definitions (8)

  • Proposition 1
  • proof
  • Proposition 2: Closed forms for Gaussian inputs
  • proof : Sketch of proof:
  • Proposition 3: Closed forms for linear Gaussian channels
  • proof : Sketch of proof:
  • Proposition 4
  • proof