Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Data-Driven Priors in the Maximum Entropy on the Mean Method for Linear Inverse Problems

Matthew King-Roskamp, Rustum Choksi, Tim Hoheisel

TL;DR

The paper develops a data-driven MEM framework for linear inverse problems, establishing almost-sure convergence of MEM solutions when priors are estimated empirically. By exploiting epi-convergence between log-moment generating functions and the MEM dual, it proves convergence of empirical MEM solutions $\overline{x}_{n,\varepsilon}$ to the true MEM solution $\overline{x}_{\mu}$ and derives explicit convergence rates in the quadratic-fidelity setting, including a $\mathcal{O}(n^{-1/4})$ rate in expectation for empirical priors. The approach hinges on epigraphical distances and epi-consistency of the MGF and dual objective, yielding rigorous guarantees for data-driven priors in linear inverse problems. Numerical denoising experiments on MNIST and Fashion-MNIST validate the theory and illustrate practical use, including measure-level postprocessing of the optimal prior.

Abstract

We establish the theoretical framework for implementing the maximumn entropy on the mean (MEM) method for linear inverse problems in the setting of approximate (data-driven) priors. We prove a.s. convergence for empirical means and further develop general estimates for the difference between the MEM solutions with different priors $μ$ and $ν$ based upon the epigraphical distance between their respective log-moment generating functions. These estimates allow us to establish a rate of convergence in expectation for empirical means. We illustrate our results with denoising on MNIST and Fashion-MNIST data sets.

Data-Driven Priors in the Maximum Entropy on the Mean Method for Linear Inverse Problems

TL;DR

The paper develops a data-driven MEM framework for linear inverse problems, establishing almost-sure convergence of MEM solutions when priors are estimated empirically. By exploiting epi-convergence between log-moment generating functions and the MEM dual, it proves convergence of empirical MEM solutions to the true MEM solution and derives explicit convergence rates in the quadratic-fidelity setting, including a rate in expectation for empirical priors. The approach hinges on epigraphical distances and epi-consistency of the MGF and dual objective, yielding rigorous guarantees for data-driven priors in linear inverse problems. Numerical denoising experiments on MNIST and Fashion-MNIST validate the theory and illustrate practical use, including measure-level postprocessing of the optimal prior.

Abstract

We establish the theoretical framework for implementing the maximumn entropy on the mean (MEM) method for linear inverse problems in the setting of approximate (data-driven) priors. We prove a.s. convergence for empirical means and further develop general estimates for the difference between the MEM solutions with different priors and based upon the epigraphical distance between their respective log-moment generating functions. These estimates allow us to establish a rate of convergence in expectation for empirical means. We illustrate our results with denoising on MNIST and Fashion-MNIST data sets.

Paper Structure

This paper contains 14 sections, 23 theorems, 100 equations, 13 figures.

Table of Contents

  1. Introduction
  2. Tools from convex analysis and the MEM method for solving the problem \ref{['lin-inverse-p']}
  3. Convex analysis
  4. Maximum Entropy on the Mean Problem
  5. Approximate and Empirical Priors, Random Functions, and Epi-consistency
  6. Epigraphical convergence and convergence of minimizers
  7. Epi-consistency of the empirical moment generating functions
  8. Epi-consistency of the dual objective functions
  9. Convergence of minimizers
  10. Convergence rates for quadratic fidelity
  11. Epigraphical distances
  12. Convergence Rates
  13. A statistical dependence on $\bm{n}$
  14. Numerical experiments

Key Result

Lemma 2.1

\newlabellemma:soln_exist0 For any lsc, proper, convex $g$, the primal problem (P) always has a solution.

Figures (13)

  • Figure 1: Recovery of an 8 with additive Gaussian noise
  • Figure 2: Rates and thresholding of optimal measure $Q_{n}$, for \ref{['fig:Gaus_noise_8']}
  • Figure 3: Recovery of an 8 with salt-and-pepper corruption noise
  • Figure 4: Rates and thresholding of optimal measure $Q_{n}$, for \ref{['fig:SP_noise_8']}
  • Figure 5: A specific type of failure case for small $n$.
  • ...and 8 more figures

Theorems & Definitions (42)

  • Remark 1.1: Working at the higher level of the probability distribution of the solution
  • Lemma 2.1
  • Proof 1
  • Theorem 2.3
  • Proof 2
  • Proposition 3.1
  • Corollary 3.2
  • Proof 3
  • Corollary 3.3
  • Proof 4
  • ...and 32 more