Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The noiseless limit and improved-prior limit of the maximum entropy method and their implications for the analytic continuation problem

Thomas Chuna, Nicholas Barnfield, Paul Hamann, Sebastian Schwalbe, Michael P. Friedlander, Tobias Dornheim

TL;DR

The paper analyzes the analytic continuation problem in quantum Monte Carlo by contrasting maximum entropy methods (MEM) with stochastic and data-driven priors, and derives mean squared error (MSE) expressions for the MEM estimator in the noiseless and improved-prior limits. It demonstrates that when the Bayesian prior is near the true solution, stochastic sampling effectively reduces to MEM via a mean-field (improved-prior) approximation, and that Bryan's algorithm becomes valid in the same regime due to linearity of the estimator. Numerical experiments on a double-Gaussian test problem show that improving the prior quality yields larger MSE reductions than merely reducing the data noise, and that the dual Newton MEM and Bryan's method converge in the improved-prior and noiseless limits. The work thus provides practical guidance to prioritize prior design for analytic continuation tasks and clarifies the conditions under which MEM or Bryan's algorithm are appropriate, linking stochastic methods to MEM through the improved-prior framework.

Abstract

Quantum Monte Carlo (QMC) methods are uniquely capable of providing exact simulations of quantum many-body systems. Unfortunately, the applications of a QMC simulation are limited because extracting dynamic properties requires solving the analytic continuation (AC) problem. Across the many fields that use QMC methods, there is no universally accepted analytic continuation algorithm for extracting dynamic properties, but many publications compare to the maximum entropy method. We investigate when entropy maximization is an acceptable approach. We show that stochastic sampling algorithms reduce to entropy maximization when the Bayesian prior is near to the true solution. We investigate when is Bryan's controversial optimization algorithm [Bryan, Eur. Biophys. J. 18, 165-174 (1990)] for entropy maximization (sometimes known as the maximum entropy method) appropriate to use. We show that Bryan's algorithm is appropriate when the noise is near zero or when the Bayesian prior is near to the true solution. We also investigate the mean squared error, finding a better scaling when the Bayesian prior is near the true solution than when the noise is near zero. We point to examples of improved data-driven Bayesian priors that have already leveraged this advantage. We support these results by solving the double Gaussian problem using both Bryan's algorithm and the newly formulated dual approach to entropy maximization [Chuna et al., J. Phys. A: Math. Theor. 58, 335203 (2025)].

The noiseless limit and improved-prior limit of the maximum entropy method and their implications for the analytic continuation problem

TL;DR

The paper analyzes the analytic continuation problem in quantum Monte Carlo by contrasting maximum entropy methods (MEM) with stochastic and data-driven priors, and derives mean squared error (MSE) expressions for the MEM estimator in the noiseless and improved-prior limits. It demonstrates that when the Bayesian prior is near the true solution, stochastic sampling effectively reduces to MEM via a mean-field (improved-prior) approximation, and that Bryan's algorithm becomes valid in the same regime due to linearity of the estimator. Numerical experiments on a double-Gaussian test problem show that improving the prior quality yields larger MSE reductions than merely reducing the data noise, and that the dual Newton MEM and Bryan's method converge in the improved-prior and noiseless limits. The work thus provides practical guidance to prioritize prior design for analytic continuation tasks and clarifies the conditions under which MEM or Bryan's algorithm are appropriate, linking stochastic methods to MEM through the improved-prior framework.

Abstract

Quantum Monte Carlo (QMC) methods are uniquely capable of providing exact simulations of quantum many-body systems. Unfortunately, the applications of a QMC simulation are limited because extracting dynamic properties requires solving the analytic continuation (AC) problem. Across the many fields that use QMC methods, there is no universally accepted analytic continuation algorithm for extracting dynamic properties, but many publications compare to the maximum entropy method. We investigate when entropy maximization is an acceptable approach. We show that stochastic sampling algorithms reduce to entropy maximization when the Bayesian prior is near to the true solution. We investigate when is Bryan's controversial optimization algorithm [Bryan, Eur. Biophys. J. 18, 165-174 (1990)] for entropy maximization (sometimes known as the maximum entropy method) appropriate to use. We show that Bryan's algorithm is appropriate when the noise is near zero or when the Bayesian prior is near to the true solution. We also investigate the mean squared error, finding a better scaling when the Bayesian prior is near the true solution than when the noise is near zero. We point to examples of improved data-driven Bayesian priors that have already leveraged this advantage. We support these results by solving the double Gaussian problem using both Bryan's algorithm and the newly formulated dual approach to entropy maximization [Chuna et al., J. Phys. A: Math. Theor. 58, 335203 (2025)].

Paper Structure

This paper contains 10 sections, 28 equations, 3 figures.

Table of Contents

  1. Introduction
  2. The mean squared error of the maximum entropy estimate
  3. Definition of the mean squared error in terms of the bias and variance
  4. MSE in the noiseless limit
  5. MSE in the improved-prior limit
  6. Numeric Investigation of MSE
  7. Description of problem
  8. Numeric Results
  9. Identifying MEM as a special case of stochastic methods in the improved-prior limit
  10. Conclusions

Figures (3)

  • Figure 1: Plot of the double Gaussian test problem from Goulko Goulko_PRB_2017 where the left plot is the first peak and the right plot is the second peak. For visualization we neglect the large flat middle region. From top to bottom, we improve the quality of the Bayesian prior, defined \ref{['eq:prior']}, with $c=1.0, \, 0.7, \, 0.5$. Within each plot, we present different noise levels.
  • Figure 2: Heatmap of the mean squared error (MSE) over varied noise (noise defined \ref{['eq:noise']}) and prior quality ($c$ defined \ref{['eq:prior']}) for the solution produced by the $\chi^2$-kink alg. with a Newton optimizer on the dual problem. Contour levels are also marked as a black horizontal line in the MSE colorbar indicate their values. The grid of red x's indicates the which $c$ and $\sigma_0$ where used, Python's matplotlib smoothly interpolates between these values. Notice that the y-axis is logarithmic, while the x axis is linear.
  • Figure 3: Heatmap of the average relative distance between $x_\text{dual N.}$ and $x_\text{Bryan}$, which are the solution produced by $\chi^2$-kink alg. with a Newton optimizer on the dual problem and the solution produced by the $\chi^2$-kink alg. with Bryan's modified Levenberg-Marquardt optimizer on the primal problem. In the improved-prior limit (i.e.$c \rightarrow 0$) and the noiseless limit the deviations between the estimates go to zero.