Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Proof of The TAP Free Energy for High-Dimensional Linear Regression with Spherical Priors at All Temperatures

Zhiyuan Yu, Jingbo Liu

Abstract

Approximate inference is central to Bayesian learning, with variational inference (VI) providing a scalable framework for posterior approximation. While mean-field VI often fails in high dimensions, the more refined Bethe approximation, equivalent to the Thouless-Anderson-Palmer (TAP) free energy in statistical physics, has long been conjectured to capture Bayes-optimal behavior. We prove that the TAP formula holds for Bayesian linear regression with a uniform spherical prior at all noise levels ($Δ>0$), extending the result of Qiu and Sen (2023) in the high-noise regime. Our argument constructs a ridge regression functional that dominates the TAP free energy, yielding the first rigorous analysis of the global optimizer of the non-concave TAP functional for a planted inference model at an arbitrary noise level. This verifies that TAP, rather than mean-field, is the correct variational description in this setting.

Proof of The TAP Free Energy for High-Dimensional Linear Regression with Spherical Priors at All Temperatures

Abstract

Approximate inference is central to Bayesian learning, with variational inference (VI) providing a scalable framework for posterior approximation. While mean-field VI often fails in high dimensions, the more refined Bethe approximation, equivalent to the Thouless-Anderson-Palmer (TAP) free energy in statistical physics, has long been conjectured to capture Bayes-optimal behavior. We prove that the TAP formula holds for Bayesian linear regression with a uniform spherical prior at all noise levels (), extending the result of Qiu and Sen (2023) in the high-noise regime. Our argument constructs a ridge regression functional that dominates the TAP free energy, yielding the first rigorous analysis of the global optimizer of the non-concave TAP functional for a planted inference model at an arbitrary noise level. This verifies that TAP, rather than mean-field, is the correct variational description in this setting.

Paper Structure

This paper contains 17 sections, 11 theorems, 84 equations, 2 figures.

Table of Contents

  1. INTRODUCTION
  2. PRELIMINARIES
  3. TAP Free Energy
  4. RS Formula
  5. MAIN RESULT
  6. PROOF OF THE MAIN RESULT
  7. Equivalence of i.i.d. Normal Prior and the Spherical Prior
  8. TAP Free Energy at a Ridge Solution
  9. Asymptotics of the Free Energy
  10. Global Maximum
  11. Concentration of the Free Energy
  12. NUMERICAL SIMULATION
  13. CONCLUSION
  14. PROOF OF NON-CONCAVITY
  15. VERIFICATION OF RS ASSUMPTIONS FOR THE GAUSSIAN PRIOR
  16. ...and 2 more sections

Key Result

Lemma 1

reeves2019replica Let $i^{\mathrm{RS}}(E; \Delta)$ be the RS potential defined in RS_potential, and let $\mathcal{Z}_p$ be the partition function defined in e2. Under the assumption that $P_0$ has finite fourth moment and satisfies the single-crossing property, we have:

Figures (2)

  • Figure 1: Numerical Results Under $\Delta = 0.1$, $\alpha = 0.75$.
  • Figure 2: Complete Collection of Results, Displaying the Discrepancy $D$ across All Combinations of $(\alpha,\Delta)$.

Theorems & Definitions (19)

  • Conjecture 1
  • Lemma 1
  • Theorem 2
  • proof
  • Corollary 1
  • proof
  • Lemma 3
  • proof
  • Lemma 4
  • Lemma 5
  • ...and 9 more