Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Differentiability and overlap concentration in optimal Bayesian inference

Hong-Bin Chen, Victor Issa

TL;DR

It is shown that at every interior differentiable point of the free energy of the model, the overlap concentrates at the gradient of the free energy and the minimum mean-square error converges to a related limit.

Abstract

In this short note, we consider models of optimal Bayesian inference of finite-rank tensor products. We add to the model a linear channel parametrized by $h$. We show that at every interior differentiable point $h$ of the free energy (associated with the model), the overlap concentrates at the gradient of the free energy and the minimum mean-square error converges to a related limit. In other words, the model is replica-symmetric at every differentiable point. At any signal-to-noise ratio, such points $h$ form a full-measure set (hence $h=0$ belongs to the closure of these points). For a sufficiently low signal-to-noise ratio, we show that every interior point is a differentiable point.

Differentiability and overlap concentration in optimal Bayesian inference

TL;DR

It is shown that at every interior differentiable point of the free energy of the model, the overlap concentrates at the gradient of the free energy and the minimum mean-square error converges to a related limit.

Abstract

In this short note, we consider models of optimal Bayesian inference of finite-rank tensor products. We add to the model a linear channel parametrized by . We show that at every interior differentiable point of the free energy (associated with the model), the overlap concentrates at the gradient of the free energy and the minimum mean-square error converges to a related limit. In other words, the model is replica-symmetric at every differentiable point. At any signal-to-noise ratio, such points form a full-measure set (hence belongs to the closure of these points). For a sufficiently low signal-to-noise ratio, we show that every interior point is a differentiable point.
Paper Structure (7 sections, 11 theorems, 77 equations)

This paper contains 7 sections, 11 theorems, 77 equations.

Table of Contents

  1. Introduction
  2. Setting and the main result
  3. Other related works
  4. Maximizers of variational representations
  5. Limits of minimal mean square errors
  6. Concentration of overlap
  7. Short-time regularity

Key Result

Theorem 1.1

Assume i.assume_1_support_X, i.assume_2_F_N(0,0)_cvg, and i.assume_3_concent. The function $\overline F_N$ converges pointwise everywhere on $\mathbb{R}_+\times S^{D}_+$ to the unique Lipschitz viscosity solution of with initial condition $f(0,\cdot) = \psi$. Moreover, $f$ always admits the representation by the Hopf formula: if in addition $\mathsf{H}$ is convex on $S^{D}_+$, then $f$ admits th

Theorems & Definitions (26)

  • Theorem 1.1: chen2022statistical limit of free energy
  • Remark 1.2: Almost everywhere differentiability
  • Theorem 1.3
  • proof
  • proof : Proof of Theorem \ref{['t.cvgF_N_to_f']}
  • Proposition 2.1: Properties of maximizers
  • Remark 2.2: Alternative Hopf--Lax formula and results
  • Remark 2.3
  • proof : Proof of Proposition \ref{['p.maximizers']}
  • Proposition 3.1: Limit of MMSE
  • ...and 16 more