Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quantum Maximum Likelihood Prediction via Hilbert Space Embeddings

Sreejith Sreekumar, Nir Weinberger

TL;DR

This work model training as learning an embedding of probability distributions into the space of quantum density operators, and in-context learning as maximum-likelihood prediction over a specified class of quantum models, and provides an interpretation of this predictor in terms of quantum reverse information projection and quantum Pythagorean theorem.

Abstract

Recent works have proposed various explanations for the ability of modern large language models (LLMs) to perform in-context prediction. We propose an alternative conceptual viewpoint from an information-geometric and statistical perspective. Motivated by Bach[2023], we model training as learning an embedding of probability distributions into the space of quantum density operators, and in-context learning as maximum-likelihood prediction over a specified class of quantum models. We provide an interpretation of this predictor in terms of quantum reverse information projection and quantum Pythagorean theorem when the class of quantum models is sufficiently expressive. We further derive non-asymptotic performance guarantees in terms of convergence rates and concentration inequalities, both in trace norm and quantum relative entropy. Our approach provides a unified framework to handle both classical and quantum LLMs.

Quantum Maximum Likelihood Prediction via Hilbert Space Embeddings

TL;DR

This work model training as learning an embedding of probability distributions into the space of quantum density operators, and in-context learning as maximum-likelihood prediction over a specified class of quantum models, and provides an interpretation of this predictor in terms of quantum reverse information projection and quantum Pythagorean theorem.

Abstract

Recent works have proposed various explanations for the ability of modern large language models (LLMs) to perform in-context prediction. We propose an alternative conceptual viewpoint from an information-geometric and statistical perspective. Motivated by Bach[2023], we model training as learning an embedding of probability distributions into the space of quantum density operators, and in-context learning as maximum-likelihood prediction over a specified class of quantum models. We provide an interpretation of this predictor in terms of quantum reverse information projection and quantum Pythagorean theorem when the class of quantum models is sufficiently expressive. We further derive non-asymptotic performance guarantees in terms of convergence rates and concentration inequalities, both in trace norm and quantum relative entropy. Our approach provides a unified framework to handle both classical and quantum LLMs.
Paper Structure (20 sections, 5 theorems, 102 equations, 1 figure)

This paper contains 20 sections, 5 theorems, 102 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related work
  4. Preliminaries
  5. Notation
  6. Quantum relative entropy and measured relative entropy
  7. Prediction task
  8. Embeddings and Quantum Maximum Likelihood Predictor
  9. Main Results
  10. Geometric Properties of Quantum Maximum Likelihood Predictor
  11. Statistical Properties of Quantum Maximum Likelihood Predictor
  12. Proofs
  13. Proof of Proposition \ref{['prop:QMLPrel']}
  14. Proof of Theorem \ref{['Thm:quantPyth']}
  15. Interpretation of Proposition \ref{['prop:QMLPrel']} in terms of Quantum Pythagorean Theorem
  16. ...and 5 more sections

Key Result

Proposition 1

Let $\rho\in \mathcal{S}(\mathbb{H})$, $\mathcal{E}_{\rho}$ be any orthonormal eigenbasis of $\rho$, and $\Sigma \subseteq \mathcal{S}(\mathbb{H})$ be a non-empty set. If $\mathcal{J}\left(\mathcal{E}_{\rho},\Sigma\right) \subseteq \Sigma$, then If $\Sigma$ is also unitarily invariant, then the above terms also equal $\inf_{\sigma \in \Sigma} \mathsf{D}_\mathsf{KL}\left(\lambda_{\rho}\middle\|\la

Figures (1)

  • Figure 1: The Quantum Pythagorean theorem.

Theorems & Definitions (5)

  • Proposition 1: Relation between MLP and QMLP
  • Theorem 1: Quantum Pythagorean Theorem
  • Proposition 2: QMLP distance bound
  • Theorem 2: Consistency and concentration of QMLP
  • Theorem 3: Matrix Hoeffding and Bernstein Inequalities, see e.g., Vershynin2018HighDimensionalProbability