Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reweighted information inequalities

Jonathan Niles-Weed

Abstract

We establish a variant of the log-Sobolev and transport-information inequalities for mixture distributions. If a probability measure $π$ can be decomposed into components that individually satisfy such inequalities, then any measure $μ$ close to $π$ in relative Fisher information is close in relative entropy or transport distance to a reweighted version of $π$ with the same mixture components but possibly different weights. This provides a user-friendly interpretation of Fisher information bounds for non-log-concave measures and explains phenomena observed in the analysis of Langevin Monte Carlo for multimodal distributions.

Reweighted information inequalities

Abstract

We establish a variant of the log-Sobolev and transport-information inequalities for mixture distributions. If a probability measure can be decomposed into components that individually satisfy such inequalities, then any measure close to in relative Fisher information is close in relative entropy or transport distance to a reweighted version of with the same mixture components but possibly different weights. This provides a user-friendly interpretation of Fisher information bounds for non-log-concave measures and explains phenomena observed in the analysis of Langevin Monte Carlo for multimodal distributions.
Paper Structure (7 sections, 9 theorems, 48 equations)

This paper contains 7 sections, 9 theorems, 48 equations.

Table of Contents

  1. Introduction
  2. Preliminaries and prior work
  3. Prior work
  4. Notation
  5. Main results
  6. Metastability and exponentially fast convergence to a reweighted mixture
  7. Duality theory and a dynamic characterization of transport-information inequalities

Key Result

Theorem 3.1

Let $\pi_1, \dots, \pi_m$ and $\pi$ satisfy assume:main. If $\pi_1, \dots, \pi_m$ satisfy eq:ti, then

Theorems & Definitions (21)

  • Definition 2.2
  • Definition 2.3
  • Definition 2.4
  • Theorem 3.1
  • proof
  • Remark 3.2
  • Theorem 3.3
  • proof
  • Remark 3.4
  • Corollary 3.5
  • ...and 11 more